[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:19:04 dist-test-slave-mjjr krb5kdc[9857](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:19:04 dist-test-slave-mjjr krb5kdc[9857](info): set up 2 sockets
Aug 11 17:19:04 dist-test-slave-mjjr krb5kdc[9857](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 17:19:05.429467 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.476s user 0.003s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:19:05 dist-test-slave-mjjr krb5kdc[9857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:19:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:19:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:19:05.575361 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:37699
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:45365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:37699
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:05.709761 9874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:05.710090 9874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:05.710172 9874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:05.714388 9874 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:19:05.714461 9874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:05.714483 9874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:05.714500 9874 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:19:05.714567 9874 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:19:05.720507 9874 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:45365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:37699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:37699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:05.722172 9874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:05.723608 9874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:05.737850 9880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:05.737982 9883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:05.738073 9881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:05.738636 9874 server_base.cc:1034] running on GCE node
I20260811 17:19:05.739539 9874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:05.741586 9874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:05.742794 9874 hybrid_clock.cc:648] HybridClock initialized: now 1786468745742767 us; error 43 us; skew 500 ppm
Aug 11 17:19:05 dist-test-slave-mjjr krb5kdc[9857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468745, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:05.746006 9874 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:19:05.747453 9874 webserver.cc:460] Webserver started at http://127.9.157.190:45989/ using document root <none> and password file <none>
I20260811 17:19:05.748522 9874 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:05.748631 9874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:05.749230 9874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:05.752385 9874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "144ac4dcac3449708cce84a5fa48ce28"
format_stamp: "Formatted at 2026-08-11 17:19:05 on dist-test-slave-mjjr"
server_key: "5b289a51632295f65f682d7c96f01d0a"
server_key_iv: "24e447c78c0fbb6d1e14292a84691344"
server_key_version: "encryptionkey@0"
I20260811 17:19:05.753329 9874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "144ac4dcac3449708cce84a5fa48ce28"
format_stamp: "Formatted at 2026-08-11 17:19:05 on dist-test-slave-mjjr"
server_key: "5b289a51632295f65f682d7c96f01d0a"
server_key_iv: "24e447c78c0fbb6d1e14292a84691344"
server_key_version: "encryptionkey@0"
I20260811 17:19:05.758862 9874 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 17:19:05.762871 9891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:05.764716 9874 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 17:19:05.764972 9874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "144ac4dcac3449708cce84a5fa48ce28"
format_stamp: "Formatted at 2026-08-11 17:19:05 on dist-test-slave-mjjr"
server_key: "5b289a51632295f65f682d7c96f01d0a"
server_key_iv: "24e447c78c0fbb6d1e14292a84691344"
server_key_version: "encryptionkey@0"
I20260811 17:19:05.765148 9874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:05.785128 9874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:05.789561 9874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:05.789853 9874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:05.799939 9874 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:37699
I20260811 17:19:05.800005 9983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:37699 every 8 connection(s)
I20260811 17:19:05.801433 9874 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:19:05.804797 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 9874
I20260811 17:19:05.804978 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:19:05.806559 9984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:05.807083 9846 external_mini_cluster.cc:1428] Setting key 7102b07b4908bfdc75420756bcda3720
I20260811 17:19:05.816766 9984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 144ac4dcac3449708cce84a5fa48ce28: Bootstrap starting.
I20260811 17:19:05.821444 9984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 144ac4dcac3449708cce84a5fa48ce28: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:05.822734 9984 log.cc:824] T 00000000000000000000000000000000 P 144ac4dcac3449708cce84a5fa48ce28: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:05.826031 9984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 144ac4dcac3449708cce84a5fa48ce28: No bootstrap required, opened a new log
Aug 11 17:19:05 dist-test-slave-mjjr krb5kdc[9857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:05.832032 9984 raft_consensus.cc:348] T 00000000000000000000000000000000 P 144ac4dcac3449708cce84a5fa48ce28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144ac4dcac3449708cce84a5fa48ce28" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37699 } }
I20260811 17:19:05.832508 9984 raft_consensus.cc:374] T 00000000000000000000000000000000 P 144ac4dcac3449708cce84a5fa48ce28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:05.832635 9984 raft_consensus.cc:729] T 00000000000000000000000000000000 P 144ac4dcac3449708cce84a5fa48ce28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 144ac4dcac3449708cce84a5fa48ce28, State: Initialized, Role: FOLLOWER
I20260811 17:19:05.833636 9984 consensus_queue.cc:260] T 00000000000000000000000000000000 P 144ac4dcac3449708cce84a5fa48ce28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144ac4dcac3449708cce84a5fa48ce28" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37699 } }
I20260811 17:19:05.833909 9984 raft_consensus.cc:388] T 00000000000000000000000000000000 P 144ac4dcac3449708cce84a5fa48ce28 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:19:05.834054 9984 raft_consensus.cc:482] T 00000000000000000000000000000000 P 144ac4dcac3449708cce84a5fa48ce28 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:19:05.834232 9984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 144ac4dcac3449708cce84a5fa48ce28 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:05.835431 9984 raft_consensus.cc:504] T 00000000000000000000000000000000 P 144ac4dcac3449708cce84a5fa48ce28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144ac4dcac3449708cce84a5fa48ce28" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37699 } }
I20260811 17:19:05.836144 9984 leader_election.cc:302] T 00000000000000000000000000000000 P 144ac4dcac3449708cce84a5fa48ce28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 144ac4dcac3449708cce84a5fa48ce28; no voters:
I20260811 17:19:05.836661 9984 leader_election.cc:288] T 00000000000000000000000000000000 P 144ac4dcac3449708cce84a5fa48ce28 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:19:05.836897 9990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 144ac4dcac3449708cce84a5fa48ce28 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:05.837231 9990 raft_consensus.cc:686] T 00000000000000000000000000000000 P 144ac4dcac3449708cce84a5fa48ce28 [term 1 LEADER]: Becoming Leader. State: Replica: 144ac4dcac3449708cce84a5fa48ce28, State: Running, Role: LEADER
I20260811 17:19:05.837761 9990 consensus_queue.cc:237] T 00000000000000000000000000000000 P 144ac4dcac3449708cce84a5fa48ce28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144ac4dcac3449708cce84a5fa48ce28" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37699 } }
I20260811 17:19:05.838681 9984 sys_catalog.cc:564] T 00000000000000000000000000000000 P 144ac4dcac3449708cce84a5fa48ce28 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:19:05.840832 9992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 144ac4dcac3449708cce84a5fa48ce28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 144ac4dcac3449708cce84a5fa48ce28. Latest consensus state: current_term: 1 leader_uuid: "144ac4dcac3449708cce84a5fa48ce28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144ac4dcac3449708cce84a5fa48ce28" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37699 } } }
I20260811 17:19:05.840998 9992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 144ac4dcac3449708cce84a5fa48ce28 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:05.841254 9991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 144ac4dcac3449708cce84a5fa48ce28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "144ac4dcac3449708cce84a5fa48ce28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "144ac4dcac3449708cce84a5fa48ce28" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37699 } } }
I20260811 17:19:05.841383 9991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 144ac4dcac3449708cce84a5fa48ce28 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:05.841794 10011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:19:05.843183 9986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:05.813073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57290 (local address 127.9.157.190:37699)
0811 17:19:05.813726 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:05.813749 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:05.815058 (+ 1309us) server_negotiation.cc:407] Connection header received
0811 17:19:05.819475 (+ 4417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:05.819504 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:05.820199 (+ 695us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:05.820628 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:05.823461 (+ 2833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:05.825082 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:05.826694 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:05.827541 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:05.827870 (+ 329us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:05.833381 (+ 5511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:05.833434 (+ 53us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:05.833464 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:05.833509 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:05.836039 (+ 2530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:05.836955 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:05.836963 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:05.836971 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:05.837166 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:05.837741 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:05.837745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:05.837747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:05.838538 (+ 791us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:05.838775 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:05.841477 (+ 2702us) server_negotiation.cc:299] Negotiation successful
0811 17:19:05.841902 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":195,"threads_started":1}
I20260811 17:19:05.848807 10011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:19:05.857977 10011 catalog_manager.cc:1347] Generated new cluster ID: 980c4a17c901452aa02aaeba8533b729
I20260811 17:19:05.858074 10011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:19:05.886057 10011 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:19:05.887591 10011 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:19:05.892601 10011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 144ac4dcac3449708cce84a5fa48ce28: Generated new TSK 0
I20260811 17:19:05.893613 10011 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
I20260811 17:19:05.933127 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:0
--local_ip_for_outbound_sockets=127.9.157.129
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:37699
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:45365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:06.062335 10027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:06.062652 10027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:06.062700 10027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:06.066987 10027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:06.067070 10027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:06.067171 10027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:19:06.073362 10027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:45365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:37699
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:06.074695 10027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:06.075922 10027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:06.087178 10039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:06.087208 10037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:06.087178 10041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:06.087718 10027 server_base.cc:1034] running on GCE node
I20260811 17:19:06.088253 10027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:06.088960 10027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:06.090193 10027 hybrid_clock.cc:648] HybridClock initialized: now 1786468746090176 us; error 33 us; skew 500 ppm
Aug 11 17:19:06 dist-test-slave-mjjr krb5kdc[9857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468746, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:06.093529 10027 init.cc:377] Logged in from keytab as kudu/127.9.157.129@KRBTEST.COM (short username kudu)
I20260811 17:19:06.094674 10027 webserver.cc:460] Webserver started at http://127.9.157.129:44223/ using document root <none> and password file <none>
I20260811 17:19:06.095340 10027 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:06.095448 10027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:06.095788 10027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:06.098153 10027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-0/data/instance:
uuid: "1afe16ccf8984581bbf9c0386b464bcf"
format_stamp: "Formatted at 2026-08-11 17:19:06 on dist-test-slave-mjjr"
server_key: "22e3c535bf40319492f18078dac98b7a"
server_key_iv: "d0a836335b097f297a31aae27425f520"
server_key_version: "encryptionkey@0"
I20260811 17:19:06.098887 10027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance:
uuid: "1afe16ccf8984581bbf9c0386b464bcf"
format_stamp: "Formatted at 2026-08-11 17:19:06 on dist-test-slave-mjjr"
server_key: "22e3c535bf40319492f18078dac98b7a"
server_key_iv: "d0a836335b097f297a31aae27425f520"
server_key_version: "encryptionkey@0"
I20260811 17:19:06.103521 10027 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:19:06.106407 10051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:06.107860 10027 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 17:19:06.108021 10027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "1afe16ccf8984581bbf9c0386b464bcf"
format_stamp: "Formatted at 2026-08-11 17:19:06 on dist-test-slave-mjjr"
server_key: "22e3c535bf40319492f18078dac98b7a"
server_key_iv: "d0a836335b097f297a31aae27425f520"
server_key_version: "encryptionkey@0"
I20260811 17:19:06.108152 10027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:06.121793 10027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:06.124876 10027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:06.125113 10027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:06.125873 10027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:06.127085 10027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:06.127164 10027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:06.127241 10027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:06.127285 10027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:06.138499 10027 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:45743
I20260811 17:19:06.138516 10221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:45743 every 8 connection(s)
I20260811 17:19:06.139552 10027 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:19:06.141783 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 10027
I20260811 17:19:06.141870 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance
I20260811 17:19:06.142195 9846 external_mini_cluster.cc:1428] Setting key 08c9ef1f956a1bbeb8dbaa52f0e3a150
Aug 11 17:19:06 dist-test-slave-mjjr krb5kdc[9857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468746, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.157.130@KRBTEST.COM; defaulting to no policy
I20260811 17:19:06.156134 9986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:06.142171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:38725 (local address 127.9.157.190:37699)
0811 17:19:06.142351 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:06.142356 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:06.143301 (+ 945us) server_negotiation.cc:407] Connection header received
0811 17:19:06.145128 (+ 1827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:06.145135 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:06.145199 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:06.145295 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:06.146538 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:06.147082 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:06.148496 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:06.148741 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:06.148865 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:06.152146 (+ 3281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:06.152149 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:06.152151 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:06.152179 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:06.153800 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:06.154429 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:06.154432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:06.154433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:06.154516 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:06.154843 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:06.154846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:06.154847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:06.155042 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:06.155116 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:06.155807 (+ 691us) server_negotiation.cc:299] Negotiation successful
0811 17:19:06.155927 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.130@KRBTEST.COM" created.
I20260811 17:19:06.157155 10226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:06.142619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:37699 (local address 127.9.157.129:38725)
0811 17:19:06.143134 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:06.143190 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:06.144897 (+ 1707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:06.145473 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:06.145480 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:06.145843 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:06.146365 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:06.146380 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:06.147219 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:06.147222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:06.148353 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:06.148371 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:06.148906 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:06.148910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:06.149096 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:06.149735 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:06.149753 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:06.151945 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:06.153925 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:06.153931 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:06.153946 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:06.154291 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:06.154600 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:06.154602 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:06.154604 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:06.154753 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:06.155132 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:06.155138 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:06.155469 (+ 331us) client_negotiation.cc:769] Sending connection context
0811 17:19:06.155736 (+ 267us) client_negotiation.cc:240] Negotiation successful
0811 17:19:06.156122 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":154,"threads_started":1}
I20260811 17:19:06.158707 10222 heartbeater.cc:347] Connected to a master server at 127.9.157.190:37699
I20260811 17:19:06.158996 10222 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:06.159564 10222 heartbeater.cc:511] Master 127.9.157.190:37699 requested a full tablet report, sending...
I20260811 17:19:06.161888 9916 ts_manager.cc:194] Registered new tserver with Master: 1afe16ccf8984581bbf9c0386b464bcf (127.9.157.129:45743)
I20260811 17:19:06.163398 9916 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.129@KRBTEST.COM'} at 127.9.157.129:38725
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
I20260811 17:19:06.175098 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:0
--local_ip_for_outbound_sockets=127.9.157.130
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:37699
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:45365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:06.301028 10229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:06.301360 10229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:06.301412 10229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:06.305833 10229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:06.305934 10229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:06.306003 10229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:19:06.311887 10229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:45365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:37699
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:06.313225 10229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:06.314316 10229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:06.324921 10243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:06.324976 10236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:06.324993 10237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:06.325172 10229 server_base.cc:1034] running on GCE node
I20260811 17:19:06.325786 10229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:06.326548 10229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:06.327724 10229 hybrid_clock.cc:648] HybridClock initialized: now 1786468746327698 us; error 31 us; skew 500 ppm
Aug 11 17:19:06 dist-test-slave-mjjr krb5kdc[9857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468746, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:06.330756 10229 init.cc:377] Logged in from keytab as kudu/127.9.157.130@KRBTEST.COM (short username kudu)
I20260811 17:19:06.331823 10229 webserver.cc:460] Webserver started at http://127.9.157.130:32957/ using document root <none> and password file <none>
I20260811 17:19:06.332463 10229 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:06.332563 10229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:06.332887 10229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:06.335242 10229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-1/data/instance:
uuid: "3da8ecf5ded44f44ac74afa17c8b403f"
format_stamp: "Formatted at 2026-08-11 17:19:06 on dist-test-slave-mjjr"
server_key: "c50f42da4837292fc096711fd6764458"
server_key_iv: "e5ac0816097dd942158dac74f8ca690f"
server_key_version: "encryptionkey@0"
I20260811 17:19:06.335969 10229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance:
uuid: "3da8ecf5ded44f44ac74afa17c8b403f"
format_stamp: "Formatted at 2026-08-11 17:19:06 on dist-test-slave-mjjr"
server_key: "c50f42da4837292fc096711fd6764458"
server_key_iv: "e5ac0816097dd942158dac74f8ca690f"
server_key_version: "encryptionkey@0"
I20260811 17:19:06.340406 10229 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 17:19:06.343194 10256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:06.344656 10229 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:06.344796 10229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "3da8ecf5ded44f44ac74afa17c8b403f"
format_stamp: "Formatted at 2026-08-11 17:19:06 on dist-test-slave-mjjr"
server_key: "c50f42da4837292fc096711fd6764458"
server_key_iv: "e5ac0816097dd942158dac74f8ca690f"
server_key_version: "encryptionkey@0"
I20260811 17:19:06.344918 10229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:06.363615 10229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:06.366797 10229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:06.367025 10229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:06.367746 10229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:06.368696 10229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:06.368745 10229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:06.368815 10229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:06.368850 10229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:06.379386 10229 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:35945
I20260811 17:19:06.379403 10432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:35945 every 8 connection(s)
I20260811 17:19:06.380522 10229 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:19:06.383800 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 10229
I20260811 17:19:06.383927 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance
I20260811 17:19:06.384222 9846 external_mini_cluster.cc:1428] Setting key ef2568f0621d0305eabc5b35fc5c6e72
Aug 11 17:19:06 dist-test-slave-mjjr krb5kdc[9857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468746, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.157.131@KRBTEST.COM; defaulting to no policy
I20260811 17:19:06.396535 9986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:06.382743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:37083 (local address 127.9.157.190:37699)
0811 17:19:06.382902 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:06.382905 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:06.383751 (+ 846us) server_negotiation.cc:407] Connection header received
0811 17:19:06.385338 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:06.385342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:06.385410 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:06.385511 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:06.386801 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:06.387337 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:06.389203 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:06.389544 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:06.389675 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:06.392578 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:06.392582 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:06.392584 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:06.392610 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:06.394112 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:06.394714 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:06.394717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:06.394718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:06.394795 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:06.395191 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:06.395195 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:06.395196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:06.395426 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:06.395535 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:06.396204 (+ 669us) server_negotiation.cc:299] Negotiation successful
0811 17:19:06.396340 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 17:19:06.397418 10435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:06.383087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:37699 (local address 127.9.157.130:37083)
0811 17:19:06.383568 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:06.383612 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:06.385110 (+ 1498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:06.385716 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:06.385724 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:06.386104 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:06.386642 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:06.386657 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:06.387484 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:06.387487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:06.389045 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:06.389060 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:06.389717 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:06.389720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:06.389900 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:06.390574 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:06.390592 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:06.392387 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:06.394256 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:06.394263 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:06.394280 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:06.394565 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:06.394920 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:06.394924 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:06.394925 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:06.395083 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:06.395547 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:06.395555 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:06.395925 (+ 370us) client_negotiation.cc:769] Sending connection context
0811 17:19:06.396141 (+ 216us) client_negotiation.cc:240] Negotiation successful
0811 17:19:06.396505 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":124,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.131@KRBTEST.COM" created.
I20260811 17:19:06.398804 10433 heartbeater.cc:347] Connected to a master server at 127.9.157.190:37699
I20260811 17:19:06.399080 10433 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:06.399627 10433 heartbeater.cc:511] Master 127.9.157.190:37699 requested a full tablet report, sending...
I20260811 17:19:06.400836 9916 ts_manager.cc:194] Registered new tserver with Master: 3da8ecf5ded44f44ac74afa17c8b403f (127.9.157.130:35945)
I20260811 17:19:06.401449 9916 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.130@KRBTEST.COM'} at 127.9.157.130:37083
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
I20260811 17:19:06.415362 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:0
--local_ip_for_outbound_sockets=127.9.157.131
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:37699
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:45365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:06.536535 10440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:06.536813 10440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:06.536859 10440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:06.541074 10440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:06.541137 10440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:06.541199 10440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:19:06.546732 10440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:45365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:37699
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:06.547880 10440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:06.549049 10440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:06.559458 10459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:06.559463 10447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:06.559542 10450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:06.560067 10440 server_base.cc:1034] running on GCE node
I20260811 17:19:06.560654 10440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:06.561631 10440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:06.562824 10440 hybrid_clock.cc:648] HybridClock initialized: now 1786468746562793 us; error 50 us; skew 500 ppm
Aug 11 17:19:06 dist-test-slave-mjjr krb5kdc[9857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468746, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:06.565778 10440 init.cc:377] Logged in from keytab as kudu/127.9.157.131@KRBTEST.COM (short username kudu)
I20260811 17:19:06.566825 10440 webserver.cc:460] Webserver started at http://127.9.157.131:45737/ using document root <none> and password file <none>
I20260811 17:19:06.567457 10440 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:06.567557 10440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:06.567837 10440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:06.570118 10440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-2/data/instance:
uuid: "e555e785505c4bb7bb1676e3bcf21b1f"
format_stamp: "Formatted at 2026-08-11 17:19:06 on dist-test-slave-mjjr"
server_key: "ea4128b6e27a7b36fed2f635ec6fe83b"
server_key_iv: "08c1c8f75a7e60624e72e86a80fc4cc8"
server_key_version: "encryptionkey@0"
I20260811 17:19:06.570809 10440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance:
uuid: "e555e785505c4bb7bb1676e3bcf21b1f"
format_stamp: "Formatted at 2026-08-11 17:19:06 on dist-test-slave-mjjr"
server_key: "ea4128b6e27a7b36fed2f635ec6fe83b"
server_key_iv: "08c1c8f75a7e60624e72e86a80fc4cc8"
server_key_version: "encryptionkey@0"
I20260811 17:19:06.574937 10440 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 17:19:06.577548 10471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:06.578864 10440 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 17:19:06.579027 10440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "e555e785505c4bb7bb1676e3bcf21b1f"
format_stamp: "Formatted at 2026-08-11 17:19:06 on dist-test-slave-mjjr"
server_key: "ea4128b6e27a7b36fed2f635ec6fe83b"
server_key_iv: "08c1c8f75a7e60624e72e86a80fc4cc8"
server_key_version: "encryptionkey@0"
I20260811 17:19:06.579156 10440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:06.615465 10440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:06.619191 10440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:06.619437 10440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:06.620276 10440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:06.621207 10440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:06.621289 10440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:06.621357 10440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:06.621407 10440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:06.631812 10440 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:35103
I20260811 17:19:06.631841 10669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:35103 every 8 connection(s)
I20260811 17:19:06.632808 10440 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:19:06.633571 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 10440
I20260811 17:19:06.633738 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance
I20260811 17:19:06.634020 9846 external_mini_cluster.cc:1428] Setting key c06b029cc850511cd4f8dc1fc645c211
Aug 11 17:19:06 dist-test-slave-mjjr krb5kdc[9857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468746, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:06.649544 9986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:06.635042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:50189 (local address 127.9.157.190:37699)
0811 17:19:06.635195 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:06.635202 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:06.637128 (+ 1926us) server_negotiation.cc:407] Connection header received
0811 17:19:06.638543 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:06.638546 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:06.638595 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:06.638666 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:06.640104 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:06.640840 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:06.642171 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:06.642409 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:06.642517 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:06.645248 (+ 2731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:06.645251 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:06.645253 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:06.645279 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:06.646776 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:06.647300 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:06.647303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:06.647304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:06.647462 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:06.647955 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:06.647978 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:06.647982 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:06.648270 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:06.648390 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:06.649242 (+ 852us) server_negotiation.cc:299] Negotiation successful
0811 17:19:06.649371 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 17:19:06.650482 10672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:06.635544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:37699 (local address 127.9.157.131:50189)
0811 17:19:06.636934 (+ 1390us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:06.636997 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:06.638334 (+ 1337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:06.638808 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:06.638815 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:06.639188 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:06.639893 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:06.639952 (+ 59us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:06.641039 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:06.641044 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:06.642020 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:06.642030 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:06.642568 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:06.642571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:06.642724 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:06.643418 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:06.643436 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:06.645058 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:06.646900 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:06.646906 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:06.646923 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:06.647197 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:06.647562 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:06.647564 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:06.647565 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:06.647828 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:06.648562 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:06.648573 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:06.648966 (+ 393us) client_negotiation.cc:769] Sending connection context
0811 17:19:06.649194 (+ 228us) client_negotiation.cc:240] Negotiation successful
0811 17:19:06.649583 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":161,"threads_started":1}
I20260811 17:19:06.651621 10670 heartbeater.cc:347] Connected to a master server at 127.9.157.190:37699
I20260811 17:19:06.651963 10670 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:06.652518 10670 heartbeater.cc:511] Master 127.9.157.190:37699 requested a full tablet report, sending...
I20260811 17:19:06.653581 9916 ts_manager.cc:194] Registered new tserver with Master: e555e785505c4bb7bb1676e3bcf21b1f (127.9.157.131:35103)
I20260811 17:19:06.654179 9916 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.131@KRBTEST.COM'} at 127.9.157.131:50189
I20260811 17:19:06.661661 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 17:19:06 dist-test-slave-mjjr krb5kdc[9857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468746, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 17:19:06 dist-test-slave-mjjr krb5kdc[9857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468746, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.157.129@KRBTEST.COM
I20260811 17:19:06.691215 10685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:06.681527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37454 (local address 127.9.157.129:45743)
0811 17:19:06.681944 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:06.681951 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:06.681975 (+ 24us) server_negotiation.cc:407] Connection header received
0811 17:19:06.682032 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:06.682037 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:06.682230 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:06.682354 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:06.683127 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:06.684047 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:06.684849 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:06.685226 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:06.685308 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:06.687860 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:06.687882 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:06.687896 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:06.687922 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:06.689503 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:06.689927 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:06.689932 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:06.689934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:06.690016 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:06.690253 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:06.690255 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:06.690256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:06.690499 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:06.690665 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:06.690819 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 17:19:06.691009 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":68,"threads_started":1}
W20260811 17:19:06.692518 10090 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37454
Aug 11 17:19:06 dist-test-slave-mjjr krb5kdc[9857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 17:19:06 dist-test-slave-mjjr krb5kdc[9857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.129@KRBTEST.COM
I20260811 17:19:06.721437 10685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:06.712807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37470 (local address 127.9.157.129:45743)
0811 17:19:06.712993 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:06.712999 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:06.713209 (+ 210us) server_negotiation.cc:407] Connection header received
0811 17:19:06.713305 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:06.713308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:06.713359 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:06.713434 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:06.714158 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:06.714705 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:06.715499 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:06.715810 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:06.715899 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:06.718332 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:06.718336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:06.718338 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:06.718377 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:06.719784 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:06.720195 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:06.720197 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:06.720199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:06.720276 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:06.720519 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:06.720521 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:06.720522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:06.720694 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:06.720797 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:06.721048 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 17:19:06.721253 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 17:19:06.722755 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 10027
I20260811 17:19:06.734591 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 10229
I20260811 17:19:06.748090 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 10440
I20260811 17:19:06.772667 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 9874
2026-08-11T17:19:06Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2864 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:19:06 dist-test-slave-mjjr krb5kdc[10694](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:19:06 dist-test-slave-mjjr krb5kdc[10694](info): set up 2 sockets
Aug 11 17:19:06 dist-test-slave-mjjr krb5kdc[10694](info): commencing operation
krb5kdc: starting...
W20260811 17:19:08.034047 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:19:08 dist-test-slave-mjjr krb5kdc[10694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:19:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:19:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:19:08.175158 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:34621
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:33205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:34621
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:08.297842 10715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:08.298267 10715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:08.298316 10715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:08.302678 10715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:19:08.302752 10715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:08.302772 10715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:08.302790 10715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:19:08.302806 10715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:19:08.308449 10715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:34621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:34621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:08.309617 10715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:08.310670 10715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:08.320082 10727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:08.320082 10728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:08.320112 10731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:08.320307 10715 server_base.cc:1034] running on GCE node
I20260811 17:19:08.321139 10715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:08.322340 10715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:08.323536 10715 hybrid_clock.cc:648] HybridClock initialized: now 1786468748323520 us; error 33 us; skew 500 ppm
Aug 11 17:19:08 dist-test-slave-mjjr krb5kdc[10694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468748, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:08.326683 10715 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:19:08.327937 10715 webserver.cc:460] Webserver started at http://127.9.157.190:44779/ using document root <none> and password file <none>
I20260811 17:19:08.328620 10715 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:08.328724 10715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:08.328994 10715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:08.331298 10715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "45aa83b99b6849419dd4df057bebddfa"
format_stamp: "Formatted at 2026-08-11 17:19:08 on dist-test-slave-mjjr"
server_key: "6f67c447ec42465d4617503c62a24d40"
server_key_iv: "a7d3e9209fa67e5fdf8e12739bb39c67"
server_key_version: "encryptionkey@0"
I20260811 17:19:08.331970 10715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "45aa83b99b6849419dd4df057bebddfa"
format_stamp: "Formatted at 2026-08-11 17:19:08 on dist-test-slave-mjjr"
server_key: "6f67c447ec42465d4617503c62a24d40"
server_key_iv: "a7d3e9209fa67e5fdf8e12739bb39c67"
server_key_version: "encryptionkey@0"
I20260811 17:19:08.336354 10715 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260811 17:19:08.339171 10742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:08.340720 10715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:08.340991 10715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "45aa83b99b6849419dd4df057bebddfa"
format_stamp: "Formatted at 2026-08-11 17:19:08 on dist-test-slave-mjjr"
server_key: "6f67c447ec42465d4617503c62a24d40"
server_key_iv: "a7d3e9209fa67e5fdf8e12739bb39c67"
server_key_version: "encryptionkey@0"
I20260811 17:19:08.341132 10715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:08.352797 10715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:08.356318 10715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:08.356541 10715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:08.364235 10715 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:34621
I20260811 17:19:08.364264 10823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:34621 every 8 connection(s)
I20260811 17:19:08.365289 10715 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:19:08.368589 10825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:08.373911 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 10715
I20260811 17:19:08.374053 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:19:08.374320 9846 external_mini_cluster.cc:1428] Setting key 454dee6dc6686c776c3d7a164888676a
I20260811 17:19:08.375151 10825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45aa83b99b6849419dd4df057bebddfa: Bootstrap starting.
I20260811 17:19:08.377854 10825 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45aa83b99b6849419dd4df057bebddfa: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:08.378688 10825 log.cc:824] T 00000000000000000000000000000000 P 45aa83b99b6849419dd4df057bebddfa: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:08.380803 10825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45aa83b99b6849419dd4df057bebddfa: No bootstrap required, opened a new log
Aug 11 17:19:08 dist-test-slave-mjjr krb5kdc[10694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:08.385154 10825 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45aa83b99b6849419dd4df057bebddfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45aa83b99b6849419dd4df057bebddfa" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34621 } }
I20260811 17:19:08.385411 10825 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45aa83b99b6849419dd4df057bebddfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:08.385500 10825 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45aa83b99b6849419dd4df057bebddfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45aa83b99b6849419dd4df057bebddfa, State: Initialized, Role: FOLLOWER
I20260811 17:19:08.386000 10825 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45aa83b99b6849419dd4df057bebddfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45aa83b99b6849419dd4df057bebddfa" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34621 } }
I20260811 17:19:08.386138 10825 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45aa83b99b6849419dd4df057bebddfa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:19:08.386236 10825 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45aa83b99b6849419dd4df057bebddfa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:19:08.386377 10825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45aa83b99b6849419dd4df057bebddfa [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:08.387715 10825 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45aa83b99b6849419dd4df057bebddfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45aa83b99b6849419dd4df057bebddfa" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34621 } }
I20260811 17:19:08.388106 10825 leader_election.cc:302] T 00000000000000000000000000000000 P 45aa83b99b6849419dd4df057bebddfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45aa83b99b6849419dd4df057bebddfa; no voters:
I20260811 17:19:08.388478 10825 leader_election.cc:288] T 00000000000000000000000000000000 P 45aa83b99b6849419dd4df057bebddfa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:19:08.388592 10831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45aa83b99b6849419dd4df057bebddfa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:08.388865 10831 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45aa83b99b6849419dd4df057bebddfa [term 1 LEADER]: Becoming Leader. State: Replica: 45aa83b99b6849419dd4df057bebddfa, State: Running, Role: LEADER
I20260811 17:19:08.389269 10831 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45aa83b99b6849419dd4df057bebddfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45aa83b99b6849419dd4df057bebddfa" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34621 } }
I20260811 17:19:08.389961 10825 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45aa83b99b6849419dd4df057bebddfa [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:19:08.390479 10828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:08.375965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41380 (local address 127.9.157.190:34621)
0811 17:19:08.376458 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:08.376474 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:08.376511 (+ 37us) server_negotiation.cc:407] Connection header received
0811 17:19:08.377192 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:08.377210 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:08.377571 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:08.377931 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:08.378789 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:08.379906 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:08.380771 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:08.381312 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:08.381503 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:08.384531 (+ 3028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:08.384537 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:08.384553 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:08.384586 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:08.386711 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:08.387223 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:08.387228 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:08.387233 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:08.387350 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:08.387711 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:08.387714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:08.387716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:08.388241 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:08.388501 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:08.389151 (+ 650us) server_negotiation.cc:299] Negotiation successful
0811 17:19:08.389463 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":153,"threads_started":1}
I20260811 17:19:08.391135 10832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45aa83b99b6849419dd4df057bebddfa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45aa83b99b6849419dd4df057bebddfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45aa83b99b6849419dd4df057bebddfa" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34621 } } }
I20260811 17:19:08.391287 10832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45aa83b99b6849419dd4df057bebddfa [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:08.391134 10834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45aa83b99b6849419dd4df057bebddfa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45aa83b99b6849419dd4df057bebddfa. Latest consensus state: current_term: 1 leader_uuid: "45aa83b99b6849419dd4df057bebddfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45aa83b99b6849419dd4df057bebddfa" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34621 } } }
I20260811 17:19:08.391397 10834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45aa83b99b6849419dd4df057bebddfa [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:08.391918 10847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:19:08.395232 10847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:19:08.400833 10847 catalog_manager.cc:1347] Generated new cluster ID: 9981fb8806b44130a71c381b5f1dd746
I20260811 17:19:08.400924 10847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:19:08.407917 10847 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:19:08.408984 10847 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:19:08.418464 10847 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45aa83b99b6849419dd4df057bebddfa: Generated new TSK 0
I20260811 17:19:08.419329 10847 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
I20260811 17:19:08.456538 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:0
--local_ip_for_outbound_sockets=127.9.157.129
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:34621
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:33205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:08.578150 10868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:08.578457 10868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:08.578544 10868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:08.582758 10868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:08.582832 10868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:08.582929 10868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:19:08.588672 10868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:34621
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:08.589924 10868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:08.591169 10868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:08.601583 10881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:08.601583 10878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:08.601593 10879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:08.601900 10868 server_base.cc:1034] running on GCE node
I20260811 17:19:08.602473 10868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:08.603209 10868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:08.604403 10868 hybrid_clock.cc:648] HybridClock initialized: now 1786468748604372 us; error 49 us; skew 500 ppm
Aug 11 17:19:08 dist-test-slave-mjjr krb5kdc[10694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468748, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:08.607290 10868 init.cc:377] Logged in from keytab as kudu/127.9.157.129@KRBTEST.COM (short username kudu)
I20260811 17:19:08.608393 10868 webserver.cc:460] Webserver started at http://127.9.157.129:34393/ using document root <none> and password file <none>
I20260811 17:19:08.609015 10868 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:08.609112 10868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:08.609360 10868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:08.611725 10868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-0/data/instance:
uuid: "c37541faebff46ab81cf00fc110a71b0"
format_stamp: "Formatted at 2026-08-11 17:19:08 on dist-test-slave-mjjr"
server_key: "0969b444061fb785f36bfea4a3ff31f8"
server_key_iv: "35ea63409c1acb2baf33b44a1899f0a9"
server_key_version: "encryptionkey@0"
I20260811 17:19:08.612365 10868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance:
uuid: "c37541faebff46ab81cf00fc110a71b0"
format_stamp: "Formatted at 2026-08-11 17:19:08 on dist-test-slave-mjjr"
server_key: "0969b444061fb785f36bfea4a3ff31f8"
server_key_iv: "35ea63409c1acb2baf33b44a1899f0a9"
server_key_version: "encryptionkey@0"
I20260811 17:19:08.616608 10868 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:19:08.619349 10894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:08.620906 10868 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 17:19:08.621078 10868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "c37541faebff46ab81cf00fc110a71b0"
format_stamp: "Formatted at 2026-08-11 17:19:08 on dist-test-slave-mjjr"
server_key: "0969b444061fb785f36bfea4a3ff31f8"
server_key_iv: "35ea63409c1acb2baf33b44a1899f0a9"
server_key_version: "encryptionkey@0"
I20260811 17:19:08.621213 10868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:08.646617 10868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:08.649971 10868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:08.650206 10868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:08.650856 10868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:08.651839 10868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:08.651911 10868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:08.651984 10868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:08.652027 10868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:08.662256 10868 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:38687
I20260811 17:19:08.662274 11070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:38687 every 8 connection(s)
I20260811 17:19:08.663259 10868 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:19:08.664685 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 10868
I20260811 17:19:08.664788 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance
I20260811 17:19:08.665060 9846 external_mini_cluster.cc:1428] Setting key 23439e6e2c359dafd941d48e89d51bd2
Aug 11 17:19:08 dist-test-slave-mjjr krb5kdc[10694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468748, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.157.130@KRBTEST.COM; defaulting to no policy
I20260811 17:19:08.678658 10828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:08.665462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:40011 (local address 127.9.157.190:34621)
0811 17:19:08.665643 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:08.665649 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:08.666778 (+ 1129us) server_negotiation.cc:407] Connection header received
0811 17:19:08.668016 (+ 1238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:08.668023 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:08.668089 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:08.668190 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:08.669456 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:08.670312 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:08.671494 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:08.671983 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:08.672119 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:08.674732 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:08.674738 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:08.674742 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:08.674776 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:08.676483 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:08.676997 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:08.677000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:08.677001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:08.677080 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:08.677503 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:08.677505 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:08.677506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:08.677699 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:08.677781 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:08.678329 (+ 548us) server_negotiation.cc:299] Negotiation successful
0811 17:19:08.678459 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.130@KRBTEST.COM" created.
I20260811 17:19:08.679725 11076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:08.665961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34621 (local address 127.9.157.129:40011)
0811 17:19:08.666538 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:08.666639 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:08.667790 (+ 1151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:08.668359 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:08.668366 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:08.668752 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:08.669288 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:08.669303 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:08.670452 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:08.670455 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:08.671354 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:08.671364 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:08.672159 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:08.672163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:08.672322 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:08.672979 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:08.672997 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:08.674525 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:08.676613 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:08.676619 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:08.676633 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:08.676899 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:08.677215 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:08.677221 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:08.677226 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:08.677415 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:08.677795 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:08.677801 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:08.678066 (+ 265us) client_negotiation.cc:769] Sending connection context
0811 17:19:08.678234 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 17:19:08.678630 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":146,"threads_started":1}
I20260811 17:19:08.681131 11071 heartbeater.cc:347] Connected to a master server at 127.9.157.190:34621
I20260811 17:19:08.681416 11071 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:08.681978 11071 heartbeater.cc:511] Master 127.9.157.190:34621 requested a full tablet report, sending...
I20260811 17:19:08.683877 10771 ts_manager.cc:194] Registered new tserver with Master: c37541faebff46ab81cf00fc110a71b0 (127.9.157.129:38687)
I20260811 17:19:08.685699 10771 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.129@KRBTEST.COM'} at 127.9.157.129:40011
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
I20260811 17:19:08.698530 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:0
--local_ip_for_outbound_sockets=127.9.157.130
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:34621
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:33205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:08.819031 11080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:08.819360 11080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:08.819416 11080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:08.823602 11080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:08.823740 11080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:08.823805 11080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:19:08.829663 11080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:34621
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:08.830864 11080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:08.831969 11080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:08.842155 11094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:08.842111 11096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:08.842160 11091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:08.842316 11080 server_base.cc:1034] running on GCE node
I20260811 17:19:08.842962 11080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:08.843721 11080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:08.844913 11080 hybrid_clock.cc:648] HybridClock initialized: now 1786468748844897 us; error 36 us; skew 500 ppm
Aug 11 17:19:08 dist-test-slave-mjjr krb5kdc[10694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468748, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:08.847882 11080 init.cc:377] Logged in from keytab as kudu/127.9.157.130@KRBTEST.COM (short username kudu)
I20260811 17:19:08.848815 11080 webserver.cc:460] Webserver started at http://127.9.157.130:38667/ using document root <none> and password file <none>
I20260811 17:19:08.849428 11080 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:08.849519 11080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:08.849759 11080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:08.851934 11080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-1/data/instance:
uuid: "c0b343b6ae604ab2b814879d24d14a02"
format_stamp: "Formatted at 2026-08-11 17:19:08 on dist-test-slave-mjjr"
server_key: "bc342948b407158605709da9c87ccd71"
server_key_iv: "9877a9edf232f979d6d9783853d79da4"
server_key_version: "encryptionkey@0"
I20260811 17:19:08.852499 11080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance:
uuid: "c0b343b6ae604ab2b814879d24d14a02"
format_stamp: "Formatted at 2026-08-11 17:19:08 on dist-test-slave-mjjr"
server_key: "bc342948b407158605709da9c87ccd71"
server_key_iv: "9877a9edf232f979d6d9783853d79da4"
server_key_version: "encryptionkey@0"
I20260811 17:19:08.856556 11080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 17:19:08.859163 11104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:08.860445 11080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 17:19:08.860594 11080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "c0b343b6ae604ab2b814879d24d14a02"
format_stamp: "Formatted at 2026-08-11 17:19:08 on dist-test-slave-mjjr"
server_key: "bc342948b407158605709da9c87ccd71"
server_key_iv: "9877a9edf232f979d6d9783853d79da4"
server_key_version: "encryptionkey@0"
I20260811 17:19:08.860714 11080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:08.871964 11080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:08.875154 11080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:08.875375 11080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:08.876133 11080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:08.877027 11080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:08.877075 11080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:08.877116 11080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:08.877141 11080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:08.886571 11080 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:34811
I20260811 17:19:08.886592 11296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:34811 every 8 connection(s)
I20260811 17:19:08.887545 11080 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:19:08.896700 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 11080
I20260811 17:19:08.896854 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance
I20260811 17:19:08.897117 9846 external_mini_cluster.cc:1428] Setting key 961e03629e2d3fac2f5ab783e256e75b
Aug 11 17:19:08 dist-test-slave-mjjr krb5kdc[10694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468748, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:08.902240 10828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:08.889565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:51073 (local address 127.9.157.190:34621)
0811 17:19:08.889710 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:08.889715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:08.890828 (+ 1113us) server_negotiation.cc:407] Connection header received
0811 17:19:08.891936 (+ 1108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:08.891939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:08.892008 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:08.892105 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:08.893344 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:08.893889 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:08.895092 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:08.895323 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:08.895440 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:08.898208 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:08.898212 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:08.898214 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:08.898249 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:08.899912 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:08.900462 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:08.900468 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:08.900473 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:08.900572 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:08.900974 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:08.900979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:08.900983 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:08.901216 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:08.901297 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:08.901997 (+ 700us) server_negotiation.cc:299] Negotiation successful
0811 17:19:08.902109 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 17:19:08.903683 11304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:08.890080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34621 (local address 127.9.157.130:51073)
0811 17:19:08.890673 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:08.890732 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:08.891698 (+ 966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:08.892294 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:08.892302 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:08.892691 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:08.893187 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:08.893201 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:08.894027 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:08.894031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:08.894954 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:08.894965 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:08.895493 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:08.895497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:08.895745 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:08.896332 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:08.896349 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:08.898027 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:08.900050 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:08.900056 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:08.900071 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:08.900357 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:08.900669 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:08.900671 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:08.900673 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:08.900833 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:08.901327 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:08.901336 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:08.901701 (+ 365us) client_negotiation.cc:769] Sending connection context
0811 17:19:08.901959 (+ 258us) client_negotiation.cc:240] Negotiation successful
0811 17:19:08.902366 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":135,"threads_started":1}
I20260811 17:19:08.905047 11297 heartbeater.cc:347] Connected to a master server at 127.9.157.190:34621
I20260811 17:19:08.905313 11297 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:08.905859 11297 heartbeater.cc:511] Master 127.9.157.190:34621 requested a full tablet report, sending...
I20260811 17:19:08.907081 10771 ts_manager.cc:194] Registered new tserver with Master: c0b343b6ae604ab2b814879d24d14a02 (127.9.157.130:34811)
I20260811 17:19:08.907763 10771 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.130@KRBTEST.COM'} at 127.9.157.130:51073
WARNING: no policy specified for kudu/127.9.157.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
I20260811 17:19:08.930899 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:0
--local_ip_for_outbound_sockets=127.9.157.131
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:34621
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:33205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:09.055423 11309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:09.055740 11309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:09.055789 11309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:09.059999 11309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:09.060079 11309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:09.060145 11309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:19:09.065853 11309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:34621
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:09.067009 11309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:09.068078 11309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:09.078179 11322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:09.078167 11320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:09.078188 11319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:09.078439 11309 server_base.cc:1034] running on GCE node
I20260811 17:19:09.079034 11309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:09.079825 11309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:09.081045 11309 hybrid_clock.cc:648] HybridClock initialized: now 1786468749081025 us; error 29 us; skew 500 ppm
Aug 11 17:19:09 dist-test-slave-mjjr krb5kdc[10694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468749, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:09.087800 11309 init.cc:377] Logged in from keytab as kudu/127.9.157.131@KRBTEST.COM (short username kudu)
I20260811 17:19:09.088989 11309 webserver.cc:460] Webserver started at http://127.9.157.131:38143/ using document root <none> and password file <none>
I20260811 17:19:09.089636 11309 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:09.089815 11309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:09.090081 11309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:09.092543 11309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-2/data/instance:
uuid: "0d16523f3682485f9beaaf93ccb661b5"
format_stamp: "Formatted at 2026-08-11 17:19:09 on dist-test-slave-mjjr"
server_key: "5d062cdf0422768650417cce0562df31"
server_key_iv: "75d732683e7643414dbbf96b7490250e"
server_key_version: "encryptionkey@0"
I20260811 17:19:09.093153 11309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance:
uuid: "0d16523f3682485f9beaaf93ccb661b5"
format_stamp: "Formatted at 2026-08-11 17:19:09 on dist-test-slave-mjjr"
server_key: "5d062cdf0422768650417cce0562df31"
server_key_iv: "75d732683e7643414dbbf96b7490250e"
server_key_version: "encryptionkey@0"
I20260811 17:19:09.097175 11309 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:19:09.099819 11336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:09.101126 11309 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 17:19:09.101269 11309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "0d16523f3682485f9beaaf93ccb661b5"
format_stamp: "Formatted at 2026-08-11 17:19:09 on dist-test-slave-mjjr"
server_key: "5d062cdf0422768650417cce0562df31"
server_key_iv: "75d732683e7643414dbbf96b7490250e"
server_key_version: "encryptionkey@0"
I20260811 17:19:09.101393 11309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:09.139007 11309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:09.143056 11309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:09.143307 11309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:09.144158 11309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:09.145277 11309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:09.145334 11309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:09.145387 11309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:09.145403 11309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:09.155853 11309 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:44939
I20260811 17:19:09.155874 11549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:44939 every 8 connection(s)
I20260811 17:19:09.156924 11309 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:19:09.160790 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 11309
I20260811 17:19:09.160948 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance
I20260811 17:19:09.161324 9846 external_mini_cluster.cc:1428] Setting key 772c06f52e085cac7a6b56e42f48f51b
Aug 11 17:19:09 dist-test-slave-mjjr krb5kdc[10694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468749, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:09.171442 10828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:09.159057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:37045 (local address 127.9.157.190:34621)
0811 17:19:09.159224 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:09.159229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:09.160219 (+ 990us) server_negotiation.cc:407] Connection header received
0811 17:19:09.161215 (+ 996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:09.161219 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:09.161283 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:09.161402 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:09.162727 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.163316 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.164722 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.164945 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.165060 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:09.167918 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:09.167926 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:09.167933 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:09.167970 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:09.169533 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:09.170020 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:09.170023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:09.170024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:09.170102 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:09.170423 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:09.170425 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:09.170426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:09.170606 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:09.170681 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:09.171180 (+ 499us) server_negotiation.cc:299] Negotiation successful
0811 17:19:09.171288 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 17:19:09.172407 11555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:09.159508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34621 (local address 127.9.157.131:37045)
0811 17:19:09.160040 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:09.160088 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:09.160960 (+ 872us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:09.161604 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:09.161611 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:09.162039 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:09.162606 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:09.162621 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.163563 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.163567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:09.164610 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:09.164620 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.165085 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.165088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:09.165324 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:09.166096 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:09.166114 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:09.167720 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:09.169639 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:09.169644 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:09.169657 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:09.169923 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:09.170177 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:09.170179 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:09.170181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:09.170317 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:09.170692 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:09.170697 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:09.170958 (+ 261us) client_negotiation.cc:769] Sending connection context
0811 17:19:09.171116 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 17:19:09.171409 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":139,"threads_started":1}
I20260811 17:19:09.173650 11553 heartbeater.cc:347] Connected to a master server at 127.9.157.190:34621
I20260811 17:19:09.173918 11553 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:09.174460 11553 heartbeater.cc:511] Master 127.9.157.190:34621 requested a full tablet report, sending...
I20260811 17:19:09.175741 10765 ts_manager.cc:194] Registered new tserver with Master: 0d16523f3682485f9beaaf93ccb661b5 (127.9.157.131:44939)
I20260811 17:19:09.176326 10765 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.131@KRBTEST.COM'} at 127.9.157.131:37045
I20260811 17:19:09.188040 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:19:09.201058 10828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:09.193064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41392 (local address 127.9.157.190:34621)
0811 17:19:09.193278 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:09.193284 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:09.193450 (+ 166us) server_negotiation.cc:407] Connection header received
0811 17:19:09.193628 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:09.193632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:09.193688 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:09.193779 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:09.194524 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.195062 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.195961 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.196197 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.196311 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:09.197727 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:09.197730 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:09.197733 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:09.197769 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:09.199222 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:09.199711 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:09.199714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:09.199716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:09.199798 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:09.200096 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:09.200099 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:09.200101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:09.200346 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:09.200503 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:09.200753 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 17:19:09.200879 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 17:19:09.207916 10765 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41392:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:19:09.211479 10765 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:19:09.228276 11572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:09.221830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:34811 (local address 127.0.0.1:35144)
0811 17:19:09.222286 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:09.222343 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:09.222485 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:09.223686 (+ 1201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:09.223692 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:09.223713 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:09.223871 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:09.223881 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.225319 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.225323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:09.226609 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:09.226619 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.227697 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.227701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:09.227810 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:09.227855 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 17:19:09.227966 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 17:19:09.228096 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":35,"thread_start_us":148,"threads_started":1}
I20260811 17:19:09.228281 11573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:09.222059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35144 (local address 127.9.157.130:34811)
0811 17:19:09.223085 (+ 1026us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:09.223098 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:09.223123 (+ 25us) server_negotiation.cc:407] Connection header received
0811 17:19:09.223211 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:09.223218 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:09.223506 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:09.223690 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:09.224005 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.225157 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.226729 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.227540 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.227685 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:09.227930 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:09.227997 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 17:19:09.228098 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":909,"thread_start_us":112,"threads_started":1}
I20260811 17:19:09.228720 11579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:09.222296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:44939 (local address 127.0.0.1:35590)
0811 17:19:09.223282 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:09.223296 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:09.223506 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:09.224208 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:09.224212 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:09.224244 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:09.224379 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:09.224387 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.225632 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.225634 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:09.226866 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:09.226877 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.228194 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.228198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:09.228307 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:09.228338 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 17:19:09.228504 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 17:19:09.228578 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":927,"mutex_wait_us":2,"spinlock_wait_cycles":8960,"thread_start_us":126,"threads_started":1}
I20260811 17:19:09.229056 11581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:09.222861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56348 (local address 127.9.157.129:38687)
0811 17:19:09.223256 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:09.223263 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:09.223473 (+ 210us) server_negotiation.cc:407] Connection header received
0811 17:19:09.223633 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:09.223639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:09.223851 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:09.224058 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:09.224550 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.226106 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.227352 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.228094 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.228194 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:09.228330 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:09.228798 (+ 468us) server_negotiation.cc:299] Negotiation successful
0811 17:19:09.228920 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":117,"threads_started":1}
I20260811 17:19:09.229329 11580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:09.222759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35590 (local address 127.9.157.131:44939)
0811 17:19:09.223189 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:09.223207 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:09.223542 (+ 335us) server_negotiation.cc:407] Connection header received
0811 17:19:09.223632 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:09.223659 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:09.223859 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:09.224055 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:09.224526 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.225482 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.227012 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.227886 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.227994 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:09.228124 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:09.228802 (+ 678us) server_negotiation.cc:299] Negotiation successful
0811 17:19:09.229152 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":109,"threads_started":1}
I20260811 17:19:09.229312 11574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:09.222653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:38687 (local address 127.0.0.1:56348)
0811 17:19:09.223356 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:09.223367 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:09.223485 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:09.224208 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:09.224212 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:09.224241 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:09.224441 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:09.224452 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.226226 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.226230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:09.227229 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:09.227237 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.228400 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.228404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:09.228527 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:09.228551 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 17:19:09.228622 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 17:19:09.228697 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"mutex_wait_us":79,"spinlock_wait_cycles":2944,"thread_start_us":103,"threads_started":1}
I20260811 17:19:09.232359 11179 tablet_service.cc:1467] Processing CreateTablet for tablet fd85f47968554536b0e02f3cef7efb00 (DEFAULT_TABLE table=test-table [id=16d49d0c94e34da48ea89d1e3c7f1a93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:09.232594 10965 tablet_service.cc:1467] Processing CreateTablet for tablet fd85f47968554536b0e02f3cef7efb00 (DEFAULT_TABLE table=test-table [id=16d49d0c94e34da48ea89d1e3c7f1a93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:09.232599 11425 tablet_service.cc:1467] Processing CreateTablet for tablet fd85f47968554536b0e02f3cef7efb00 (DEFAULT_TABLE table=test-table [id=16d49d0c94e34da48ea89d1e3c7f1a93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:09.233716 10965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd85f47968554536b0e02f3cef7efb00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:09.233717 11425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd85f47968554536b0e02f3cef7efb00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:09.233724 11179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd85f47968554536b0e02f3cef7efb00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:09.241035 11582 tablet_bootstrap.cc:492] T fd85f47968554536b0e02f3cef7efb00 P c0b343b6ae604ab2b814879d24d14a02: Bootstrap starting.
I20260811 17:19:09.241984 11589 tablet_bootstrap.cc:492] T fd85f47968554536b0e02f3cef7efb00 P 0d16523f3682485f9beaaf93ccb661b5: Bootstrap starting.
I20260811 17:19:09.243896 11589 tablet_bootstrap.cc:654] T fd85f47968554536b0e02f3cef7efb00 P 0d16523f3682485f9beaaf93ccb661b5: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:09.244138 11583 tablet_bootstrap.cc:492] T fd85f47968554536b0e02f3cef7efb00 P c37541faebff46ab81cf00fc110a71b0: Bootstrap starting.
I20260811 17:19:09.244611 11582 tablet_bootstrap.cc:654] T fd85f47968554536b0e02f3cef7efb00 P c0b343b6ae604ab2b814879d24d14a02: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:09.245469 11589 log.cc:824] T fd85f47968554536b0e02f3cef7efb00 P 0d16523f3682485f9beaaf93ccb661b5: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:09.245824 11582 log.cc:824] T fd85f47968554536b0e02f3cef7efb00 P c0b343b6ae604ab2b814879d24d14a02: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:09.246734 11583 tablet_bootstrap.cc:654] T fd85f47968554536b0e02f3cef7efb00 P c37541faebff46ab81cf00fc110a71b0: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:09.247408 11583 log.cc:824] T fd85f47968554536b0e02f3cef7efb00 P c37541faebff46ab81cf00fc110a71b0: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:09.247952 11582 tablet_bootstrap.cc:492] T fd85f47968554536b0e02f3cef7efb00 P c0b343b6ae604ab2b814879d24d14a02: No bootstrap required, opened a new log
I20260811 17:19:09.247965 11589 tablet_bootstrap.cc:492] T fd85f47968554536b0e02f3cef7efb00 P 0d16523f3682485f9beaaf93ccb661b5: No bootstrap required, opened a new log
I20260811 17:19:09.248178 11589 ts_tablet_manager.cc:1397] T fd85f47968554536b0e02f3cef7efb00 P 0d16523f3682485f9beaaf93ccb661b5: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 17:19:09.248178 11582 ts_tablet_manager.cc:1397] T fd85f47968554536b0e02f3cef7efb00 P c0b343b6ae604ab2b814879d24d14a02: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260811 17:19:09.249361 11583 tablet_bootstrap.cc:492] T fd85f47968554536b0e02f3cef7efb00 P c37541faebff46ab81cf00fc110a71b0: No bootstrap required, opened a new log
I20260811 17:19:09.249584 11583 ts_tablet_manager.cc:1397] T fd85f47968554536b0e02f3cef7efb00 P c37541faebff46ab81cf00fc110a71b0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 17:19:09.252573 11589 raft_consensus.cc:348] T fd85f47968554536b0e02f3cef7efb00 P 0d16523f3682485f9beaaf93ccb661b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b343b6ae604ab2b814879d24d14a02" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 34811 } } peers { permanent_uuid: "c37541faebff46ab81cf00fc110a71b0" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38687 } } peers { permanent_uuid: "0d16523f3682485f9beaaf93ccb661b5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44939 } }
I20260811 17:19:09.252573 11582 raft_consensus.cc:348] T fd85f47968554536b0e02f3cef7efb00 P c0b343b6ae604ab2b814879d24d14a02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b343b6ae604ab2b814879d24d14a02" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 34811 } } peers { permanent_uuid: "c37541faebff46ab81cf00fc110a71b0" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38687 } } peers { permanent_uuid: "0d16523f3682485f9beaaf93ccb661b5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44939 } }
I20260811 17:19:09.252923 11589 raft_consensus.cc:374] T fd85f47968554536b0e02f3cef7efb00 P 0d16523f3682485f9beaaf93ccb661b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:09.252923 11582 raft_consensus.cc:374] T fd85f47968554536b0e02f3cef7efb00 P c0b343b6ae604ab2b814879d24d14a02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:09.252988 11582 raft_consensus.cc:729] T fd85f47968554536b0e02f3cef7efb00 P c0b343b6ae604ab2b814879d24d14a02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0b343b6ae604ab2b814879d24d14a02, State: Initialized, Role: FOLLOWER
I20260811 17:19:09.252988 11589 raft_consensus.cc:729] T fd85f47968554536b0e02f3cef7efb00 P 0d16523f3682485f9beaaf93ccb661b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d16523f3682485f9beaaf93ccb661b5, State: Initialized, Role: FOLLOWER
I20260811 17:19:09.253510 11589 consensus_queue.cc:260] T fd85f47968554536b0e02f3cef7efb00 P 0d16523f3682485f9beaaf93ccb661b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b343b6ae604ab2b814879d24d14a02" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 34811 } } peers { permanent_uuid: "c37541faebff46ab81cf00fc110a71b0" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38687 } } peers { permanent_uuid: "0d16523f3682485f9beaaf93ccb661b5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44939 } }
I20260811 17:19:09.253510 11582 consensus_queue.cc:260] T fd85f47968554536b0e02f3cef7efb00 P c0b343b6ae604ab2b814879d24d14a02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b343b6ae604ab2b814879d24d14a02" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 34811 } } peers { permanent_uuid: "c37541faebff46ab81cf00fc110a71b0" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38687 } } peers { permanent_uuid: "0d16523f3682485f9beaaf93ccb661b5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44939 } }
I20260811 17:19:09.254348 11583 raft_consensus.cc:348] T fd85f47968554536b0e02f3cef7efb00 P c37541faebff46ab81cf00fc110a71b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b343b6ae604ab2b814879d24d14a02" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 34811 } } peers { permanent_uuid: "c37541faebff46ab81cf00fc110a71b0" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38687 } } peers { permanent_uuid: "0d16523f3682485f9beaaf93ccb661b5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44939 } }
I20260811 17:19:09.254658 11583 raft_consensus.cc:374] T fd85f47968554536b0e02f3cef7efb00 P c37541faebff46ab81cf00fc110a71b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:09.254719 11583 raft_consensus.cc:729] T fd85f47968554536b0e02f3cef7efb00 P c37541faebff46ab81cf00fc110a71b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c37541faebff46ab81cf00fc110a71b0, State: Initialized, Role: FOLLOWER
I20260811 17:19:09.254722 11582 ts_tablet_manager.cc:1428] T fd85f47968554536b0e02f3cef7efb00 P c0b343b6ae604ab2b814879d24d14a02: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260811 17:19:09.254709 11553 heartbeater.cc:503] Master 127.9.157.190:34621 was elected leader, sending a full tablet report...
I20260811 17:19:09.254750 11297 heartbeater.cc:503] Master 127.9.157.190:34621 was elected leader, sending a full tablet report...
I20260811 17:19:09.255235 11583 consensus_queue.cc:260] T fd85f47968554536b0e02f3cef7efb00 P c37541faebff46ab81cf00fc110a71b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b343b6ae604ab2b814879d24d14a02" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 34811 } } peers { permanent_uuid: "c37541faebff46ab81cf00fc110a71b0" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38687 } } peers { permanent_uuid: "0d16523f3682485f9beaaf93ccb661b5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44939 } }
I20260811 17:19:09.255996 11071 heartbeater.cc:503] Master 127.9.157.190:34621 was elected leader, sending a full tablet report...
I20260811 17:19:09.256151 11583 ts_tablet_manager.cc:1428] T fd85f47968554536b0e02f3cef7efb00 P c37541faebff46ab81cf00fc110a71b0: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 17:19:09.258533 11589 ts_tablet_manager.cc:1428] T fd85f47968554536b0e02f3cef7efb00 P 0d16523f3682485f9beaaf93ccb661b5: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20260811 17:19:09.275913 11601 raft_consensus.cc:482] T fd85f47968554536b0e02f3cef7efb00 P c37541faebff46ab81cf00fc110a71b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:19:09.276116 11601 raft_consensus.cc:504] T fd85f47968554536b0e02f3cef7efb00 P c37541faebff46ab81cf00fc110a71b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b343b6ae604ab2b814879d24d14a02" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 34811 } } peers { permanent_uuid: "c37541faebff46ab81cf00fc110a71b0" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38687 } } peers { permanent_uuid: "0d16523f3682485f9beaaf93ccb661b5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44939 } }
I20260811 17:19:09.277261 11601 leader_election.cc:288] T fd85f47968554536b0e02f3cef7efb00 P c37541faebff46ab81cf00fc110a71b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0b343b6ae604ab2b814879d24d14a02 (127.9.157.130:34811), 0d16523f3682485f9beaaf93ccb661b5 (127.9.157.131:44939)
I20260811 17:19:09.282680 11573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:09.277690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:38763 (local address 127.9.157.130:34811)
0811 17:19:09.277846 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:09.277853 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:09.278197 (+ 344us) server_negotiation.cc:407] Connection header received
0811 17:19:09.278313 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:09.278317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:09.278373 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:09.278461 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:09.278830 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.279618 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.281372 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.281994 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.282125 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:09.282180 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:09.282468 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 17:19:09.282540 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 17:19:09.282680 11580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:09.277658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:57475 (local address 127.9.157.131:44939)
0811 17:19:09.277870 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:09.277875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:09.278140 (+ 265us) server_negotiation.cc:407] Connection header received
0811 17:19:09.278313 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:09.278317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:09.278369 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:09.278461 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:09.278830 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.279618 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.281372 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.281995 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.282125 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:09.282180 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:09.282468 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 17:19:09.282540 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260811 17:19:09.282748 11603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:09.277511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:34811 (local address 127.9.157.129:38763)
0811 17:19:09.278114 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:09.278127 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:09.278206 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:09.278497 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:09.278500 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:09.278578 (+ 78us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:09.278707 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:09.278714 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.279762 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.279765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:09.281242 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:09.281254 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.282134 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.282138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:09.282310 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:09.282357 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 17:19:09.282438 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 17:19:09.282547 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"mutex_wait_us":240,"spinlock_wait_cycles":20992,"thread_start_us":160,"threads_started":1}
I20260811 17:19:09.282758 11604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:09.277508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:44939 (local address 127.9.157.129:57475)
0811 17:19:09.278043 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:09.278059 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:09.278191 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:09.278497 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:09.278500 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:09.278519 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:09.278707 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:09.278714 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.279762 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.279766 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:09.281242 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:09.281254 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.282134 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.282138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:09.282290 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:09.282353 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 17:19:09.282438 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 17:19:09.282549 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"mutex_wait_us":122,"spinlock_wait_cycles":24832,"thread_start_us":199,"threads_started":1}
I20260811 17:19:09.283634 11459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd85f47968554536b0e02f3cef7efb00" candidate_uuid: "c37541faebff46ab81cf00fc110a71b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d16523f3682485f9beaaf93ccb661b5" is_pre_election: true
I20260811 17:19:09.283715 11214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd85f47968554536b0e02f3cef7efb00" candidate_uuid: "c37541faebff46ab81cf00fc110a71b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0b343b6ae604ab2b814879d24d14a02" is_pre_election: true
I20260811 17:19:09.284192 11459 raft_consensus.cc:2454] T fd85f47968554536b0e02f3cef7efb00 P 0d16523f3682485f9beaaf93ccb661b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c37541faebff46ab81cf00fc110a71b0 in term 0.
I20260811 17:19:09.284899 10895 leader_election.cc:302] T fd85f47968554536b0e02f3cef7efb00 P c37541faebff46ab81cf00fc110a71b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d16523f3682485f9beaaf93ccb661b5, c37541faebff46ab81cf00fc110a71b0; no voters:
I20260811 17:19:09.285192 11601 raft_consensus.cc:2781] T fd85f47968554536b0e02f3cef7efb00 P c37541faebff46ab81cf00fc110a71b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:19:09.285287 11601 raft_consensus.cc:482] T fd85f47968554536b0e02f3cef7efb00 P c37541faebff46ab81cf00fc110a71b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:19:09.285367 11601 raft_consensus.cc:3037] T fd85f47968554536b0e02f3cef7efb00 P c37541faebff46ab81cf00fc110a71b0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:09.286500 11214 raft_consensus.cc:2454] T fd85f47968554536b0e02f3cef7efb00 P c0b343b6ae604ab2b814879d24d14a02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c37541faebff46ab81cf00fc110a71b0 in term 0.
I20260811 17:19:09.286499 11601 raft_consensus.cc:504] T fd85f47968554536b0e02f3cef7efb00 P c37541faebff46ab81cf00fc110a71b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b343b6ae604ab2b814879d24d14a02" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 34811 } } peers { permanent_uuid: "c37541faebff46ab81cf00fc110a71b0" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38687 } } peers { permanent_uuid: "0d16523f3682485f9beaaf93ccb661b5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44939 } }
I20260811 17:19:09.286904 11601 leader_election.cc:288] T fd85f47968554536b0e02f3cef7efb00 P c37541faebff46ab81cf00fc110a71b0 [CANDIDATE]: Term 1 election: Requested vote from peers c0b343b6ae604ab2b814879d24d14a02 (127.9.157.130:34811), 0d16523f3682485f9beaaf93ccb661b5 (127.9.157.131:44939)
I20260811 17:19:09.287297 11459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd85f47968554536b0e02f3cef7efb00" candidate_uuid: "c37541faebff46ab81cf00fc110a71b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d16523f3682485f9beaaf93ccb661b5"
I20260811 17:19:09.287412 11459 raft_consensus.cc:3037] T fd85f47968554536b0e02f3cef7efb00 P 0d16523f3682485f9beaaf93ccb661b5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:09.287560 11214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd85f47968554536b0e02f3cef7efb00" candidate_uuid: "c37541faebff46ab81cf00fc110a71b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0b343b6ae604ab2b814879d24d14a02"
I20260811 17:19:09.287724 11214 raft_consensus.cc:3037] T fd85f47968554536b0e02f3cef7efb00 P c0b343b6ae604ab2b814879d24d14a02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:09.288632 11459 raft_consensus.cc:2454] T fd85f47968554536b0e02f3cef7efb00 P 0d16523f3682485f9beaaf93ccb661b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c37541faebff46ab81cf00fc110a71b0 in term 1.
I20260811 17:19:09.288918 11214 raft_consensus.cc:2454] T fd85f47968554536b0e02f3cef7efb00 P c0b343b6ae604ab2b814879d24d14a02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c37541faebff46ab81cf00fc110a71b0 in term 1.
I20260811 17:19:09.289027 10895 leader_election.cc:302] T fd85f47968554536b0e02f3cef7efb00 P c37541faebff46ab81cf00fc110a71b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d16523f3682485f9beaaf93ccb661b5, c37541faebff46ab81cf00fc110a71b0; no voters:
I20260811 17:19:09.289278 11601 raft_consensus.cc:2781] T fd85f47968554536b0e02f3cef7efb00 P c37541faebff46ab81cf00fc110a71b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:09.289536 11601 raft_consensus.cc:686] T fd85f47968554536b0e02f3cef7efb00 P c37541faebff46ab81cf00fc110a71b0 [term 1 LEADER]: Becoming Leader. State: Replica: c37541faebff46ab81cf00fc110a71b0, State: Running, Role: LEADER
I20260811 17:19:09.289969 11601 consensus_queue.cc:237] T fd85f47968554536b0e02f3cef7efb00 P c37541faebff46ab81cf00fc110a71b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b343b6ae604ab2b814879d24d14a02" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 34811 } } peers { permanent_uuid: "c37541faebff46ab81cf00fc110a71b0" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38687 } } peers { permanent_uuid: "0d16523f3682485f9beaaf93ccb661b5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44939 } }
I20260811 17:19:09.293772 10765 catalog_manager.cc:5495] T fd85f47968554536b0e02f3cef7efb00 P c37541faebff46ab81cf00fc110a71b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to c37541faebff46ab81cf00fc110a71b0 (127.9.157.129). New cstate: current_term: 1 leader_uuid: "c37541faebff46ab81cf00fc110a71b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0b343b6ae604ab2b814879d24d14a02" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 34811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c37541faebff46ab81cf00fc110a71b0" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d16523f3682485f9beaaf93ccb661b5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44939 } health_report { overall_health: UNKNOWN } } }
Aug 11 17:19:09 dist-test-slave-mjjr krb5kdc[10694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.129@KRBTEST.COM
I20260811 17:19:09.311862 11581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:09.302020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56358 (local address 127.9.157.129:38687)
0811 17:19:09.302198 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:09.302205 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:09.302394 (+ 189us) server_negotiation.cc:407] Connection header received
0811 17:19:09.302576 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:09.302579 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:09.302626 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:09.302718 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:09.303477 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.304072 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.305015 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.305235 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.305343 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:09.308169 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:09.308186 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:09.308204 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:09.308233 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:09.309945 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:09.310406 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:09.310412 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:09.310413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:09.310568 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:09.310805 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:09.310807 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:09.310808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:09.311110 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:09.311271 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:09.311497 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 17:19:09.311607 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 17:19:09 dist-test-slave-mjjr krb5kdc[10694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468749, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 17:19:09 dist-test-slave-mjjr krb5kdc[10694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468749, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.157.129@KRBTEST.COM
I20260811 17:19:09.342008 11581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:09.333261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56364 (local address 127.9.157.129:38687)
0811 17:19:09.333441 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:09.333447 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:09.333602 (+ 155us) server_negotiation.cc:407] Connection header received
0811 17:19:09.333793 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:09.333796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:09.333841 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:09.333939 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:09.334654 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.335192 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.336119 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.336368 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.336482 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:09.339018 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:09.339021 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:09.339023 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:09.339049 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:09.340466 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:09.340870 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:09.340872 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:09.340874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:09.340981 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:09.341217 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:09.341219 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:09.341220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:09.341387 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:09.341489 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:09.341756 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 17:19:09.341864 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
Aug 11 17:19:09 dist-test-slave-mjjr krb5kdc[10694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 17:19:09 dist-test-slave-mjjr krb5kdc[10694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.129@KRBTEST.COM
I20260811 17:19:09.371775 11581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:09.362435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56368 (local address 127.9.157.129:38687)
0811 17:19:09.362614 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:09.362619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:09.362787 (+ 168us) server_negotiation.cc:407] Connection header received
0811 17:19:09.362978 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:09.362982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:09.363036 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:09.363127 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:09.363917 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.364545 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.365403 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:09.365675 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:09.365777 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:09.368553 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:09.368557 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:09.368559 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:09.368586 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:09.369985 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:09.370439 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:09.370443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:09.370445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:09.370532 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:09.370818 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:09.370824 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:09.370826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:09.371029 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:09.371196 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:09.371444 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 17:19:09.371570 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 17:19:09.373963 10939 tablet_service.cc:2908] Rejecting scan request for tablet fd85f47968554536b0e02f3cef7efb00: Uninitialized: clean time has not yet been initialized
I20260811 17:19:09.376878 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 10868
W20260811 17:19:09.388919 11302 tablet.cc:2369] T fd85f47968554536b0e02f3cef7efb00 P c0b343b6ae604ab2b814879d24d14a02: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:19:09.393707 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 11080
I20260811 17:19:09.407325 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 11309
I20260811 17:19:09.422271 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 10715
2026-08-11T17:19:09Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2653 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:19:09 dist-test-slave-mjjr krb5kdc[11631](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:19:09 dist-test-slave-mjjr krb5kdc[11631](info): set up 2 sockets
Aug 11 17:19:09 dist-test-slave-mjjr krb5kdc[11631](info): commencing operation
krb5kdc: starting...
W20260811 17:19:10.715345 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.257s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:19:10 dist-test-slave-mjjr krb5kdc[11631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:19:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:19:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:19:10.854632 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:41783
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:46097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:41783
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:10.975507 11658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:10.975879 11658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:10.975976 11658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:10.980070 11658 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:19:10.980158 11658 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 17:19:10.980187 11658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:10.980242 11658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:10.980295 11658 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:19:10.980335 11658 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:19:10.985788 11658 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:46097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:41783
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:41783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:10.987017 11658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:10.988155 11658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:10.996938 11669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:10.996939 11675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:10.997076 11658 server_base.cc:1034] running on GCE node
W20260811 17:19:10.997020 11672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:10.997757 11658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:10.998852 11658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:11.000020 11658 hybrid_clock.cc:648] HybridClock initialized: now 1786468750999990 us; error 42 us; skew 500 ppm
Aug 11 17:19:10 dist-test-slave-mjjr krb5kdc[11631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468750, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:11.002878 11658 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:19:11.003932 11658 webserver.cc:460] Webserver started at http://127.9.157.190:37981/ using document root <none> and password file <none>
I20260811 17:19:11.004539 11658 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:11.004603 11658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:11.004849 11658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:11.007050 11658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "60b19882c4e5479fb29e3709af7fa42a"
format_stamp: "Formatted at 2026-08-11 17:19:10 on dist-test-slave-mjjr"
server_key: "695c40cbebcc58181e476d102ceafcd0"
server_key_iv: "5cf14ec9449a056131e6cfe4f66994e6"
server_key_version: "encryptionkey@0"
I20260811 17:19:11.007710 11658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "60b19882c4e5479fb29e3709af7fa42a"
format_stamp: "Formatted at 2026-08-11 17:19:10 on dist-test-slave-mjjr"
server_key: "695c40cbebcc58181e476d102ceafcd0"
server_key_iv: "5cf14ec9449a056131e6cfe4f66994e6"
server_key_version: "encryptionkey@0"
I20260811 17:19:11.011569 11658 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 17:19:11.014043 11687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:11.015401 11658 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 17:19:11.015571 11658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "60b19882c4e5479fb29e3709af7fa42a"
format_stamp: "Formatted at 2026-08-11 17:19:10 on dist-test-slave-mjjr"
server_key: "695c40cbebcc58181e476d102ceafcd0"
server_key_iv: "5cf14ec9449a056131e6cfe4f66994e6"
server_key_version: "encryptionkey@0"
I20260811 17:19:11.015738 11658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:11.038478 11658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:11.041887 11658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:11.042105 11658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:11.051286 11658 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:41783
I20260811 17:19:11.051306 11800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:41783 every 8 connection(s)
I20260811 17:19:11.052378 11658 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:19:11.052626 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 11658
I20260811 17:19:11.052762 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:19:11.053102 9846 external_mini_cluster.cc:1428] Setting key 43766ae1c1e67232346d473a06c0d6fa
I20260811 17:19:11.055939 11801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:11.063282 11801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60b19882c4e5479fb29e3709af7fa42a: Bootstrap starting.
Aug 11 17:19:11 dist-test-slave-mjjr krb5kdc[11631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:11.066421 11801 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60b19882c4e5479fb29e3709af7fa42a: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:11.068035 11801 log.cc:824] T 00000000000000000000000000000000 P 60b19882c4e5479fb29e3709af7fa42a: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:11.069476 11809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:11.054679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45058 (local address 127.9.157.190:41783)
0811 17:19:11.055124 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:11.055144 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:11.055188 (+ 44us) server_negotiation.cc:407] Connection header received
0811 17:19:11.056120 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:11.056145 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:11.056577 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:11.056959 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:11.057723 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.059245 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.060246 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.060927 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.061152 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:11.064053 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:11.064060 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:11.064077 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:11.064112 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:11.066119 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:11.066613 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:11.066619 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:11.066625 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:11.066783 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:11.067061 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:11.067064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:11.067066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:11.067748 (+ 682us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:11.067956 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:11.068216 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 17:19:11.068550 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":110,"threads_started":1}
I20260811 17:19:11.070343 11801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60b19882c4e5479fb29e3709af7fa42a: No bootstrap required, opened a new log
I20260811 17:19:11.073799 11801 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60b19882c4e5479fb29e3709af7fa42a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60b19882c4e5479fb29e3709af7fa42a" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41783 } }
I20260811 17:19:11.074084 11801 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60b19882c4e5479fb29e3709af7fa42a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:11.074179 11801 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60b19882c4e5479fb29e3709af7fa42a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60b19882c4e5479fb29e3709af7fa42a, State: Initialized, Role: FOLLOWER
I20260811 17:19:11.074625 11801 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60b19882c4e5479fb29e3709af7fa42a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60b19882c4e5479fb29e3709af7fa42a" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41783 } }
I20260811 17:19:11.074780 11801 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60b19882c4e5479fb29e3709af7fa42a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:19:11.074860 11801 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60b19882c4e5479fb29e3709af7fa42a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:19:11.074963 11801 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60b19882c4e5479fb29e3709af7fa42a [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:11.076061 11801 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60b19882c4e5479fb29e3709af7fa42a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60b19882c4e5479fb29e3709af7fa42a" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41783 } }
I20260811 17:19:11.076421 11801 leader_election.cc:302] T 00000000000000000000000000000000 P 60b19882c4e5479fb29e3709af7fa42a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60b19882c4e5479fb29e3709af7fa42a; no voters:
I20260811 17:19:11.076774 11801 leader_election.cc:288] T 00000000000000000000000000000000 P 60b19882c4e5479fb29e3709af7fa42a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:19:11.076925 11813 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60b19882c4e5479fb29e3709af7fa42a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:11.077181 11813 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60b19882c4e5479fb29e3709af7fa42a [term 1 LEADER]: Becoming Leader. State: Replica: 60b19882c4e5479fb29e3709af7fa42a, State: Running, Role: LEADER
I20260811 17:19:11.077505 11813 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60b19882c4e5479fb29e3709af7fa42a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60b19882c4e5479fb29e3709af7fa42a" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41783 } }
I20260811 17:19:11.078104 11801 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60b19882c4e5479fb29e3709af7fa42a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:19:11.079747 11815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60b19882c4e5479fb29e3709af7fa42a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60b19882c4e5479fb29e3709af7fa42a. Latest consensus state: current_term: 1 leader_uuid: "60b19882c4e5479fb29e3709af7fa42a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60b19882c4e5479fb29e3709af7fa42a" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41783 } } }
I20260811 17:19:11.079808 11814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60b19882c4e5479fb29e3709af7fa42a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60b19882c4e5479fb29e3709af7fa42a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60b19882c4e5479fb29e3709af7fa42a" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41783 } } }
I20260811 17:19:11.079897 11815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60b19882c4e5479fb29e3709af7fa42a [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:11.079928 11814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60b19882c4e5479fb29e3709af7fa42a [sys.catalog]: This master's current role is: LEADER
W20260811 17:19:11.083002 11833 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 60b19882c4e5479fb29e3709af7fa42a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:19:11.083105 11833 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:19:11.083238 11830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:19:11.084299 11830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:19:11.091672 11830 catalog_manager.cc:1347] Generated new cluster ID: 109da3db59e040e3b11b62af1ca83ee8
I20260811 17:19:11.091778 11830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:19:11.126068 11830 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:19:11.127110 11830 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:19:11.133096 11830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60b19882c4e5479fb29e3709af7fa42a: Generated new TSK 0
I20260811 17:19:11.133860 11830 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
I20260811 17:19:11.178761 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:0
--local_ip_for_outbound_sockets=127.9.157.129
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:41783
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:46097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:11.308744 11843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:11.309055 11843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:11.309108 11843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:11.313244 11843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:11.313324 11843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:11.313367 11843 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 17:19:11.313405 11843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:19:11.318775 11843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:46097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.157.190:41783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:11.320029 11843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:11.321159 11843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:11.331456 11849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:11.331532 11850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:11.331456 11852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:11.331979 11843 server_base.cc:1034] running on GCE node
I20260811 17:19:11.332468 11843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:11.333154 11843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:11.334363 11843 hybrid_clock.cc:648] HybridClock initialized: now 1786468751334349 us; error 28 us; skew 500 ppm
Aug 11 17:19:11 dist-test-slave-mjjr krb5kdc[11631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468751, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:11.337323 11843 init.cc:377] Logged in from keytab as kudu/127.9.157.129@KRBTEST.COM (short username kudu)
I20260811 17:19:11.338296 11843 webserver.cc:460] Webserver started at http://127.9.157.129:46633/ using document root <none> and password file <none>
I20260811 17:19:11.338932 11843 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:11.339028 11843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:11.339278 11843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:11.341466 11843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-0/data/instance:
uuid: "0e518880cdf24ad68dde3b4cdad50219"
format_stamp: "Formatted at 2026-08-11 17:19:11 on dist-test-slave-mjjr"
server_key: "4553393983ee9cbd3f28625bccf61ebc"
server_key_iv: "4891260169adfd3d1e71a37d26c62c6c"
server_key_version: "encryptionkey@0"
I20260811 17:19:11.342048 11843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance:
uuid: "0e518880cdf24ad68dde3b4cdad50219"
format_stamp: "Formatted at 2026-08-11 17:19:11 on dist-test-slave-mjjr"
server_key: "4553393983ee9cbd3f28625bccf61ebc"
server_key_iv: "4891260169adfd3d1e71a37d26c62c6c"
server_key_version: "encryptionkey@0"
I20260811 17:19:11.346064 11843 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 17:19:11.348765 11862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:11.350121 11843 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:11.350286 11843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "0e518880cdf24ad68dde3b4cdad50219"
format_stamp: "Formatted at 2026-08-11 17:19:11 on dist-test-slave-mjjr"
server_key: "4553393983ee9cbd3f28625bccf61ebc"
server_key_iv: "4891260169adfd3d1e71a37d26c62c6c"
server_key_version: "encryptionkey@0"
I20260811 17:19:11.350415 11843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:11.362397 11843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:11.365651 11843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:11.365882 11843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:11.367443 11843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:11.367519 11843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:11.367586 11843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:11.367623 11843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 17:19:11 dist-test-slave-mjjr krb5kdc[11631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468751, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:11.379503 11843 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:37597
I20260811 17:19:11.379868 12052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:37597 every 8 connection(s)
I20260811 17:19:11.380749 11843 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:19:11.382652 11809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:11.368414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:49767 (local address 127.9.157.190:41783)
0811 17:19:11.368548 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:11.368552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:11.369637 (+ 1085us) server_negotiation.cc:407] Connection header received
0811 17:19:11.370665 (+ 1028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:11.370673 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:11.370749 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:11.370856 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:11.372439 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.373061 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.374320 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.374568 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.374688 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:11.378489 (+ 3801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:11.378493 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:11.378496 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:11.378573 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:11.380454 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:11.381158 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:11.381162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:11.381164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:11.381266 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:11.381647 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:11.381650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:11.381651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:11.381860 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:11.381944 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:11.382413 (+ 469us) server_negotiation.cc:299] Negotiation successful
0811 17:19:11.382530 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 17:19:11.383507 11877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:11.368987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:41783 (local address 127.9.157.129:49767)
0811 17:19:11.369483 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:11.369528 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:11.370388 (+ 860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:11.371118 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:11.371125 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:11.371575 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:11.372223 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:11.372249 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.373208 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.373212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:11.374182 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:11.374193 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.374739 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.374743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:11.375018 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:11.375721 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:11.375742 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:11.378276 (+ 2534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:11.380595 (+ 2319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:11.380600 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:11.380612 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:11.381044 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:11.381381 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:11.381385 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:11.381386 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:11.381554 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:11.381961 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:11.381966 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:11.382200 (+ 234us) client_negotiation.cc:769] Sending connection context
0811 17:19:11.382363 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 17:19:11.382617 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":135,"threads_started":1}
I20260811 17:19:11.384881 12054 heartbeater.cc:347] Connected to a master server at 127.9.157.190:41783
I20260811 17:19:11.385049 12054 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:11.385665 12054 heartbeater.cc:511] Master 127.9.157.190:41783 requested a full tablet report, sending...
I20260811 17:19:11.387270 11715 ts_manager.cc:194] Registered new tserver with Master: 0e518880cdf24ad68dde3b4cdad50219 (127.9.157.129:37597)
I20260811 17:19:11.388124 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 11843
I20260811 17:19:11.388247 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance
I20260811 17:19:11.388552 9846 external_mini_cluster.cc:1428] Setting key 6f791313a9c4b69715024871e6dc3496
I20260811 17:19:11.388770 11715 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.129@KRBTEST.COM'} at 127.9.157.129:49767
I20260811 17:19:11.394848 11809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:11.387415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:42967 (local address 127.9.157.190:41783)
0811 17:19:11.387551 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:11.387555 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:11.387821 (+ 266us) server_negotiation.cc:407] Connection header received
0811 17:19:11.387945 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:11.387948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:11.387991 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:11.388077 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:11.388632 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.389335 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.390253 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.390490 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.390598 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:11.391792 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:11.391795 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:11.391798 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:11.391829 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:11.393177 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:11.393622 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:11.393626 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:11.393628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:11.393712 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:11.393999 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:11.394004 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:11.394005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:11.394186 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:11.394293 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:11.394569 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 17:19:11.394681 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":64}
I20260811 17:19:11.394858 12065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:11.387346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:41783 (local address 127.9.157.129:42967)
0811 17:19:11.387733 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:11.387747 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:11.387854 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:11.388120 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:11.388123 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:11.388364 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:11.388506 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:11.388515 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.389452 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.389456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:11.390131 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:11.390141 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.390615 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.390618 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:11.390714 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:11.391252 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:11.391265 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:11.391608 (+ 343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:11.393300 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:11.393303 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:11.393305 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:11.393509 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:11.393812 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:11.393816 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:11.393818 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:11.393905 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:11.394291 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:11.394294 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:11.394395 (+ 101us) client_negotiation.cc:769] Sending connection context
0811 17:19:11.394501 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 17:19:11.394651 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":113,"threads_started":1}
WARNING: no policy specified for kudu/127.9.157.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
I20260811 17:19:11.419399 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:0
--local_ip_for_outbound_sockets=127.9.157.130
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:41783
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:46097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:11.538513 12069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:11.538784 12069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:11.538830 12069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:11.542888 12069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:11.542958 12069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:11.542996 12069 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 17:19:11.543030 12069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:19:11.548489 12069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:46097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.157.190:41783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:11.549536 12069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:11.550559 12069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:11.560910 12086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:11.560909 12079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:11.560910 12083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:11.561273 12069 server_base.cc:1034] running on GCE node
I20260811 17:19:11.561743 12069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:11.562485 12069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:11.563710 12069 hybrid_clock.cc:648] HybridClock initialized: now 1786468751563702 us; error 46 us; skew 500 ppm
Aug 11 17:19:11 dist-test-slave-mjjr krb5kdc[11631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468751, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:11.566658 12069 init.cc:377] Logged in from keytab as kudu/127.9.157.130@KRBTEST.COM (short username kudu)
I20260811 17:19:11.567627 12069 webserver.cc:460] Webserver started at http://127.9.157.130:46527/ using document root <none> and password file <none>
I20260811 17:19:11.568261 12069 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:11.568362 12069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:11.568602 12069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:11.570712 12069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-1/data/instance:
uuid: "1b732ab9a8764c11bc7dfab37c1e76cc"
format_stamp: "Formatted at 2026-08-11 17:19:11 on dist-test-slave-mjjr"
server_key: "a64ff84c36b13b9cf7b37e1255d81e6e"
server_key_iv: "25ef9df1fc279aa50ae9a40af5c7e175"
server_key_version: "encryptionkey@0"
I20260811 17:19:11.571307 12069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance:
uuid: "1b732ab9a8764c11bc7dfab37c1e76cc"
format_stamp: "Formatted at 2026-08-11 17:19:11 on dist-test-slave-mjjr"
server_key: "a64ff84c36b13b9cf7b37e1255d81e6e"
server_key_iv: "25ef9df1fc279aa50ae9a40af5c7e175"
server_key_version: "encryptionkey@0"
I20260811 17:19:11.575435 12069 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260811 17:19:11.578313 12101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:11.580885 12069 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 17:19:11.581032 12069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "1b732ab9a8764c11bc7dfab37c1e76cc"
format_stamp: "Formatted at 2026-08-11 17:19:11 on dist-test-slave-mjjr"
server_key: "a64ff84c36b13b9cf7b37e1255d81e6e"
server_key_iv: "25ef9df1fc279aa50ae9a40af5c7e175"
server_key_version: "encryptionkey@0"
I20260811 17:19:11.581167 12069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:11.601480 12069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:11.605000 12069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:11.605235 12069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:11.606930 12069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:11.607007 12069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:11.607074 12069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:11.607105 12069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 17:19:11 dist-test-slave-mjjr krb5kdc[11631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468751, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:11.619117 12069 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:40129
I20260811 17:19:11.619163 12305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:40129 every 8 connection(s)
I20260811 17:19:11.620190 12069 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:19:11.622488 11809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:11.607913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:59453 (local address 127.9.157.190:41783)
0811 17:19:11.608055 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:11.608058 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:11.609172 (+ 1114us) server_negotiation.cc:407] Connection header received
0811 17:19:11.610551 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:11.610558 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:11.610620 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:11.610670 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:11.612255 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.612852 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.614037 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.614331 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.614449 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:11.618112 (+ 3663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:11.618116 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:11.618118 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:11.618145 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:11.619978 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:11.620583 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:11.620586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:11.620588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:11.620700 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:11.621307 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:11.621311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:11.621313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:11.621526 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:11.621639 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:11.622166 (+ 527us) server_negotiation.cc:299] Negotiation successful
0811 17:19:11.622311 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 17:19:11.623270 12118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:11.608460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:41783 (local address 127.9.157.130:59453)
0811 17:19:11.609000 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:11.609047 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:11.610146 (+ 1099us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:11.611040 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:11.611046 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:11.611406 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:11.612067 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:11.612092 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.612989 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.612992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:11.613899 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:11.613910 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.614496 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.614501 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:11.614734 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:11.615467 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:11.615484 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:11.617915 (+ 2431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:11.620104 (+ 2189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:11.620111 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:11.620126 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:11.620465 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:11.620998 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:11.621002 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:11.621004 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:11.621180 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:11.621640 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:11.621648 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:11.621916 (+ 268us) client_negotiation.cc:769] Sending connection context
0811 17:19:11.622106 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 17:19:11.622442 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":152,"threads_started":1}
I20260811 17:19:11.624562 12307 heartbeater.cc:347] Connected to a master server at 127.9.157.190:41783
I20260811 17:19:11.624716 12307 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:11.625331 12307 heartbeater.cc:511] Master 127.9.157.190:41783 requested a full tablet report, sending...
I20260811 17:19:11.626554 11715 ts_manager.cc:194] Registered new tserver with Master: 1b732ab9a8764c11bc7dfab37c1e76cc (127.9.157.130:40129)
I20260811 17:19:11.627317 11715 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.130@KRBTEST.COM'} at 127.9.157.130:59453
I20260811 17:19:11.627683 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 12069
I20260811 17:19:11.627807 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance
I20260811 17:19:11.628142 9846 external_mini_cluster.cc:1428] Setting key 8c65d2661c9b11b6dd9954387ff23444
I20260811 17:19:11.635699 11809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:11.627105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:55925 (local address 127.9.157.190:41783)
0811 17:19:11.627244 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:11.627249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:11.627457 (+ 208us) server_negotiation.cc:407] Connection header received
0811 17:19:11.627698 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:11.627701 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:11.627746 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:11.627822 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:11.628463 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.629119 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.630196 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.630414 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.630490 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:11.631936 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:11.631939 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:11.631941 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:11.631963 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:11.633830 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:11.634341 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:11.634344 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:11.634345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:11.634418 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:11.634746 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:11.634749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:11.634750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:11.634970 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:11.635052 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:11.635427 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 17:19:11.635525 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 17:19:11.635910 12318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:11.627051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:41783 (local address 127.9.157.130:55925)
0811 17:19:11.627367 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:11.627384 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:11.627528 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:11.627942 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:11.627945 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:11.628198 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:11.628329 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:11.628337 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.629249 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.629252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:11.630055 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:11.630067 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.630534 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.630540 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:11.630670 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:11.631260 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:11.631281 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:11.631783 (+ 502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:11.633947 (+ 2164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:11.633953 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:11.633957 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:11.634248 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:11.634529 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:11.634535 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:11.634540 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:11.634648 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:11.635073 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:11.635079 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:11.635203 (+ 124us) client_negotiation.cc:769] Sending connection context
0811 17:19:11.635372 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 17:19:11.635574 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":109,"threads_started":1}
WARNING: no policy specified for kudu/127.9.157.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
I20260811 17:19:11.659057 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:0
--local_ip_for_outbound_sockets=127.9.157.131
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:41783
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:46097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:11.778285 12322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:11.778638 12322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:11.778743 12322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:11.783183 12322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:11.783265 12322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:11.783321 12322 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 17:19:11.783411 12322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:19:11.789494 12322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:46097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.157.190:41783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:11.790725 12322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:11.791882 12322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:11.802307 12328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:11.802314 12337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:11.802400 12332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:11.802543 12322 server_base.cc:1034] running on GCE node
I20260811 17:19:11.803180 12322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:11.803956 12322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:11.805130 12322 hybrid_clock.cc:648] HybridClock initialized: now 1786468751805091 us; error 49 us; skew 500 ppm
Aug 11 17:19:11 dist-test-slave-mjjr krb5kdc[11631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468751, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:11.808236 12322 init.cc:377] Logged in from keytab as kudu/127.9.157.131@KRBTEST.COM (short username kudu)
I20260811 17:19:11.809135 12322 webserver.cc:460] Webserver started at http://127.9.157.131:32789/ using document root <none> and password file <none>
I20260811 17:19:11.809695 12322 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:11.809757 12322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:11.809935 12322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:11.812201 12322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-2/data/instance:
uuid: "d8873a8068724601bb05db6d633a0ca6"
format_stamp: "Formatted at 2026-08-11 17:19:11 on dist-test-slave-mjjr"
server_key: "69dad7aa51352b6b52fb9fbad859dc4e"
server_key_iv: "be8093e4972a5d4875de3f6943105e46"
server_key_version: "encryptionkey@0"
I20260811 17:19:11.812753 12322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance:
uuid: "d8873a8068724601bb05db6d633a0ca6"
format_stamp: "Formatted at 2026-08-11 17:19:11 on dist-test-slave-mjjr"
server_key: "69dad7aa51352b6b52fb9fbad859dc4e"
server_key_iv: "be8093e4972a5d4875de3f6943105e46"
server_key_version: "encryptionkey@0"
I20260811 17:19:11.817250 12322 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 17:19:11.820513 12347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:11.822002 12322 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 17:19:11.822165 12322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "d8873a8068724601bb05db6d633a0ca6"
format_stamp: "Formatted at 2026-08-11 17:19:11 on dist-test-slave-mjjr"
server_key: "69dad7aa51352b6b52fb9fbad859dc4e"
server_key_iv: "be8093e4972a5d4875de3f6943105e46"
server_key_version: "encryptionkey@0"
I20260811 17:19:11.822304 12322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:11.833434 12322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:11.836687 12322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:11.836923 12322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:11.838490 12322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:11.838542 12322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:11.838584 12322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:11.838642 12322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 17:19:11 dist-test-slave-mjjr krb5kdc[11631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468751, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:11.850687 12322 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:39833
I20260811 17:19:11.850973 12544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:39833 every 8 connection(s)
I20260811 17:19:11.851804 12322 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:19:11.853711 11809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:11.839533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:41073 (local address 127.9.157.190:41783)
0811 17:19:11.839732 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:11.839746 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:11.840725 (+ 979us) server_negotiation.cc:407] Connection header received
0811 17:19:11.841912 (+ 1187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:11.841915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:11.841980 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:11.842103 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:11.843892 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.844774 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.846192 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.846459 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.846579 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:11.849632 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:11.849636 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:11.849639 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:11.849676 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:11.851459 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:11.852049 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:11.852053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:11.852055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:11.852146 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:11.852550 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:11.852554 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:11.852555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:11.852839 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:11.852947 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:11.853476 (+ 529us) server_negotiation.cc:299] Negotiation successful
0811 17:19:11.853587 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 17:19:11.854653 12361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:11.840046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:41783 (local address 127.9.157.131:41073)
0811 17:19:11.840542 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:11.840596 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:11.841665 (+ 1069us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:11.842355 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:11.842367 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:11.842878 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:11.843692 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:11.843719 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.844906 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.844910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:11.846041 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:11.846052 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.846659 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.846663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:11.846871 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:11.847571 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:11.847589 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:11.849443 (+ 1854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:11.851585 (+ 2142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:11.851591 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:11.851607 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:11.851946 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:11.852242 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:11.852245 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:11.852247 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:11.852428 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:11.852977 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:11.852982 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:11.853227 (+ 245us) client_negotiation.cc:769] Sending connection context
0811 17:19:11.853411 (+ 184us) client_negotiation.cc:240] Negotiation successful
0811 17:19:11.853679 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":140,"threads_started":1}
I20260811 17:19:11.856081 12545 heartbeater.cc:347] Connected to a master server at 127.9.157.190:41783
I20260811 17:19:11.856266 12545 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:11.856942 12545 heartbeater.cc:511] Master 127.9.157.190:41783 requested a full tablet report, sending...
I20260811 17:19:11.857125 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 12322
I20260811 17:19:11.857271 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance
I20260811 17:19:11.857568 9846 external_mini_cluster.cc:1428] Setting key 43f0fd807b1f014178d1b590f273f664
I20260811 17:19:11.858623 11716 ts_manager.cc:194] Registered new tserver with Master: d8873a8068724601bb05db6d633a0ca6 (127.9.157.131:39833)
I20260811 17:19:11.859436 11716 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.131@KRBTEST.COM'} at 127.9.157.131:41073
I20260811 17:19:11.860716 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:19:11.870476 11809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:11.858838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:50409 (local address 127.9.157.190:41783)
0811 17:19:11.858968 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:11.858973 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:11.859437 (+ 464us) server_negotiation.cc:407] Connection header received
0811 17:19:11.859572 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:11.859574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:11.859619 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:11.859799 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:11.860707 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.861484 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.862387 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.862811 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.862961 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:11.864624 (+ 1663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:11.864631 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:11.864635 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:11.864701 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:11.867619 (+ 2918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:11.868560 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:11.868565 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:11.868567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:11.868729 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:11.869244 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:11.869248 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:11.869250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:11.869571 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:11.869655 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:11.870117 (+ 462us) server_negotiation.cc:299] Negotiation successful
0811 17:19:11.870280 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 17:19:11.871582 12555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:11.859029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:41783 (local address 127.9.157.131:50409)
0811 17:19:11.859323 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:11.859346 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:11.859460 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:11.859936 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:11.859940 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:11.860213 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:11.860368 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:11.860379 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.861647 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.861650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:11.862256 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:11.862265 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.862955 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.862958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:11.863038 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:11.863810 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:11.863830 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:11.864395 (+ 565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:11.867882 (+ 3487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:11.867887 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:11.867890 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:11.868366 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:11.868924 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:11.868928 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:11.868930 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:11.869087 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:11.869752 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:11.869755 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:11.869869 (+ 114us) client_negotiation.cc:769] Sending connection context
0811 17:19:11.871005 (+ 1136us) client_negotiation.cc:240] Negotiation successful
0811 17:19:11.871370 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":98,"threads_started":1}
Aug 11 17:19:11 dist-test-slave-mjjr krb5kdc[11631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468751, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 17:19:11 dist-test-slave-mjjr krb5kdc[11631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468751, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:11.898522 11809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:11.889541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45062 (local address 127.9.157.190:41783)
0811 17:19:11.889695 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:11.889699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:11.889713 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:19:11.889772 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:11.889775 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:11.889821 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:11.889905 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:11.890676 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.891226 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.892158 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.892401 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.892497 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:11.895027 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:11.895031 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:11.895033 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:11.895063 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:11.896726 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:11.897203 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:11.897207 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:11.897211 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:11.897301 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:11.897589 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:11.897593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:11.897595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:11.897807 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:11.897954 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:11.898216 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 17:19:11.898342 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 17:19:11.901777 11715 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45062:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:19:11.905037 11715 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:19:11.921610 12573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:11.914767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:37597 (local address 127.0.0.1:40720)
0811 17:19:11.915439 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:11.915500 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:11.915684 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:11.916942 (+ 1258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:11.916948 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:11.916973 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:11.917118 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:11.917124 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.918578 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.918585 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:11.919976 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:11.919988 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.921141 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.921144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:11.921239 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:11.921287 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 17:19:11.921381 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 17:19:11.921479 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"mutex_wait_us":7,"thread_start_us":89,"threads_started":1}
I20260811 17:19:11.921713 12575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:11.914922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40720 (local address 127.9.157.129:37597)
0811 17:19:11.916436 (+ 1514us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:11.916443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:11.916464 (+ 21us) server_negotiation.cc:407] Connection header received
0811 17:19:11.916532 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:11.916538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:11.916769 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:11.916930 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:11.917286 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.918439 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.920122 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.920995 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.921133 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:11.921317 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:11.921416 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 17:19:11.921540 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1379,"thread_start_us":90,"threads_started":1}
I20260811 17:19:11.922495 12580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:11.915840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:40129 (local address 127.0.0.1:36162)
0811 17:19:11.917171 (+ 1331us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:11.917182 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:11.917301 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:11.917796 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:11.917799 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:11.917822 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:11.917935 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:11.917941 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.919299 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.919305 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:11.920779 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:11.920792 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.922049 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.922053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:11.922150 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:11.922176 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 17:19:11.922262 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 17:19:11.922334 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1275,"mutex_wait_us":99,"thread_start_us":135,"threads_started":1}
I20260811 17:19:11.922993 12581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:11.916069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36162 (local address 127.9.157.130:40129)
0811 17:19:11.916522 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:11.916538 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:11.917295 (+ 757us) server_negotiation.cc:407] Connection header received
0811 17:19:11.917422 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:11.917426 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:11.917631 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:11.917793 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:11.918065 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.919145 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.920928 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.921797 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.921925 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:11.922120 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:11.922712 (+ 592us) server_negotiation.cc:299] Negotiation successful
0811 17:19:11.922829 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":148,"threads_started":1}
I20260811 17:19:11.923946 12574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:11.914900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:39833 (local address 127.0.0.1:48636)
0811 17:19:11.916197 (+ 1297us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:11.916213 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:11.916332 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:11.917305 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:11.917308 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:11.917329 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:11.917429 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:11.917436 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.919314 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.919317 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:11.920779 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:11.920792 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.923437 (+ 2645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.923441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:11.923540 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:11.923567 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 17:19:11.923720 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 17:19:11.923804 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1230,"mutex_wait_us":118,"thread_start_us":95,"threads_started":1}
I20260811 17:19:11.925354 11946 tablet_service.cc:1467] Processing CreateTablet for tablet 1e8a7b650c574f8c8eab055053636370 (DEFAULT_TABLE table=test-table [id=119d8dad43e845cb8bb70656e21171df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:11.925549 12184 tablet_service.cc:1467] Processing CreateTablet for tablet 1e8a7b650c574f8c8eab055053636370 (DEFAULT_TABLE table=test-table [id=119d8dad43e845cb8bb70656e21171df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:11.926535 11946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e8a7b650c574f8c8eab055053636370. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:11.926577 12184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e8a7b650c574f8c8eab055053636370. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:11.933334 12576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:11.915522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48636 (local address 127.9.157.131:39833)
0811 17:19:11.916046 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:11.916059 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:11.916438 (+ 379us) server_negotiation.cc:407] Connection header received
0811 17:19:11.916532 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:11.916538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:11.916761 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:11.916930 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:11.917567 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.919199 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.922216 (+ 3017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.923039 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.923166 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:11.923361 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:11.932934 (+ 9573us) server_negotiation.cc:299] Negotiation successful
0811 17:19:11.933101 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":159,"threads_started":1}
I20260811 17:19:11.934181 12583 tablet_bootstrap.cc:492] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219: Bootstrap starting.
I20260811 17:19:11.936240 12585 tablet_bootstrap.cc:492] T 1e8a7b650c574f8c8eab055053636370 P 1b732ab9a8764c11bc7dfab37c1e76cc: Bootstrap starting.
I20260811 17:19:11.936831 12583 tablet_bootstrap.cc:654] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:11.936720 12436 tablet_service.cc:1467] Processing CreateTablet for tablet 1e8a7b650c574f8c8eab055053636370 (DEFAULT_TABLE table=test-table [id=119d8dad43e845cb8bb70656e21171df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:11.937898 12436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e8a7b650c574f8c8eab055053636370. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:11.937852 12583 log.cc:824] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:11.938696 12585 tablet_bootstrap.cc:654] T 1e8a7b650c574f8c8eab055053636370 P 1b732ab9a8764c11bc7dfab37c1e76cc: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:11.939625 12585 log.cc:824] T 1e8a7b650c574f8c8eab055053636370 P 1b732ab9a8764c11bc7dfab37c1e76cc: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:11.940587 12583 tablet_bootstrap.cc:492] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219: No bootstrap required, opened a new log
I20260811 17:19:11.940848 12583 ts_tablet_manager.cc:1397] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 17:19:11.941644 12585 tablet_bootstrap.cc:492] T 1e8a7b650c574f8c8eab055053636370 P 1b732ab9a8764c11bc7dfab37c1e76cc: No bootstrap required, opened a new log
I20260811 17:19:11.941880 12585 ts_tablet_manager.cc:1397] T 1e8a7b650c574f8c8eab055053636370 P 1b732ab9a8764c11bc7dfab37c1e76cc: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 17:19:11.944936 12585 raft_consensus.cc:348] T 1e8a7b650c574f8c8eab055053636370 P 1b732ab9a8764c11bc7dfab37c1e76cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } }
I20260811 17:19:11.945214 12585 raft_consensus.cc:374] T 1e8a7b650c574f8c8eab055053636370 P 1b732ab9a8764c11bc7dfab37c1e76cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:11.945273 12585 raft_consensus.cc:729] T 1e8a7b650c574f8c8eab055053636370 P 1b732ab9a8764c11bc7dfab37c1e76cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b732ab9a8764c11bc7dfab37c1e76cc, State: Initialized, Role: FOLLOWER
I20260811 17:19:11.945755 12585 consensus_queue.cc:260] T 1e8a7b650c574f8c8eab055053636370 P 1b732ab9a8764c11bc7dfab37c1e76cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } }
I20260811 17:19:11.946151 12583 raft_consensus.cc:348] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } }
I20260811 17:19:11.946465 12583 raft_consensus.cc:374] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:11.946527 12583 raft_consensus.cc:729] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e518880cdf24ad68dde3b4cdad50219, State: Initialized, Role: FOLLOWER
I20260811 17:19:11.946956 12585 ts_tablet_manager.cc:1428] T 1e8a7b650c574f8c8eab055053636370 P 1b732ab9a8764c11bc7dfab37c1e76cc: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260811 17:19:11.946995 12583 consensus_queue.cc:260] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } }
I20260811 17:19:11.947386 12307 heartbeater.cc:503] Master 127.9.157.190:41783 was elected leader, sending a full tablet report...
I20260811 17:19:11.947788 12591 tablet_bootstrap.cc:492] T 1e8a7b650c574f8c8eab055053636370 P d8873a8068724601bb05db6d633a0ca6: Bootstrap starting.
I20260811 17:19:11.948144 12583 ts_tablet_manager.cc:1428] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 17:19:11.948544 12054 heartbeater.cc:503] Master 127.9.157.190:41783 was elected leader, sending a full tablet report...
I20260811 17:19:11.950548 12591 tablet_bootstrap.cc:654] T 1e8a7b650c574f8c8eab055053636370 P d8873a8068724601bb05db6d633a0ca6: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:11.951500 12591 log.cc:824] T 1e8a7b650c574f8c8eab055053636370 P d8873a8068724601bb05db6d633a0ca6: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:11.953816 12591 tablet_bootstrap.cc:492] T 1e8a7b650c574f8c8eab055053636370 P d8873a8068724601bb05db6d633a0ca6: No bootstrap required, opened a new log
I20260811 17:19:11.954048 12591 ts_tablet_manager.cc:1397] T 1e8a7b650c574f8c8eab055053636370 P d8873a8068724601bb05db6d633a0ca6: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 17:19:11.957728 12591 raft_consensus.cc:348] T 1e8a7b650c574f8c8eab055053636370 P d8873a8068724601bb05db6d633a0ca6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } }
I20260811 17:19:11.957979 12591 raft_consensus.cc:374] T 1e8a7b650c574f8c8eab055053636370 P d8873a8068724601bb05db6d633a0ca6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:11.958052 12591 raft_consensus.cc:729] T 1e8a7b650c574f8c8eab055053636370 P d8873a8068724601bb05db6d633a0ca6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8873a8068724601bb05db6d633a0ca6, State: Initialized, Role: FOLLOWER
I20260811 17:19:11.958534 12591 consensus_queue.cc:260] T 1e8a7b650c574f8c8eab055053636370 P d8873a8068724601bb05db6d633a0ca6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } }
I20260811 17:19:11.959331 12545 heartbeater.cc:503] Master 127.9.157.190:41783 was elected leader, sending a full tablet report...
I20260811 17:19:11.959636 12591 ts_tablet_manager.cc:1428] T 1e8a7b650c574f8c8eab055053636370 P d8873a8068724601bb05db6d633a0ca6: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 17:19:11.960549 12592 raft_consensus.cc:482] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:19:11.960769 12592 raft_consensus.cc:504] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } }
I20260811 17:19:11.962020 12592 leader_election.cc:288] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8873a8068724601bb05db6d633a0ca6 (127.9.157.131:39833), 1b732ab9a8764c11bc7dfab37c1e76cc (127.9.157.130:40129)
I20260811 17:19:11.966873 12576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:11.962812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:41959 (local address 127.9.157.131:39833)
0811 17:19:11.962930 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:11.962934 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:11.962950 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:19:11.963010 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:11.963013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:11.963055 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:11.963138 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:11.963438 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.964227 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.965681 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.966283 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.966402 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:11.966472 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:11.966682 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 17:19:11.966776 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 17:19:11.966917 12600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:11.962252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:39833 (local address 127.9.157.129:41959)
0811 17:19:11.962683 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:11.962695 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:11.962835 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:11.963161 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:11.963163 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:11.963180 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:11.963287 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:11.963292 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.964340 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.964343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:11.965528 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:11.965561 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.966403 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.966406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:11.966545 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:11.966568 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 17:19:11.966662 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 17:19:11.966745 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":24,"thread_start_us":131,"threads_started":1}
I20260811 17:19:11.967769 12599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:11.962316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:40129 (local address 127.9.157.129:33755)
0811 17:19:11.963497 (+ 1181us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:11.963513 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:11.963688 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:11.963979 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:11.963982 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:11.964000 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:11.964114 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:11.964120 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.965114 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.965116 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:11.966561 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:11.966571 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.967347 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.967352 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:11.967459 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:11.967478 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 17:19:11.967541 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 17:19:11.967588 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1112,"mutex_wait_us":24,"thread_start_us":122,"threads_started":1}
I20260811 17:19:11.967804 12581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:11.962402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:33755 (local address 127.9.157.130:40129)
0811 17:19:11.962539 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:11.962544 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:11.963594 (+ 1050us) server_negotiation.cc:407] Connection header received
0811 17:19:11.963817 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:11.963821 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:11.963877 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:11.963941 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:11.964221 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.964995 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.966687 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:11.967224 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:11.967329 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:11.967387 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:11.967577 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 17:19:11.967669 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 17:19:11.967721 12472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e8a7b650c574f8c8eab055053636370" candidate_uuid: "0e518880cdf24ad68dde3b4cdad50219" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8873a8068724601bb05db6d633a0ca6" is_pre_election: true
I20260811 17:19:11.968070 12472 raft_consensus.cc:2454] T 1e8a7b650c574f8c8eab055053636370 P d8873a8068724601bb05db6d633a0ca6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e518880cdf24ad68dde3b4cdad50219 in term 0.
I20260811 17:19:11.968433 12213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e8a7b650c574f8c8eab055053636370" candidate_uuid: "0e518880cdf24ad68dde3b4cdad50219" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" is_pre_election: true
I20260811 17:19:11.968568 11867 leader_election.cc:302] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e518880cdf24ad68dde3b4cdad50219, d8873a8068724601bb05db6d633a0ca6; no voters:
I20260811 17:19:11.968758 12213 raft_consensus.cc:2454] T 1e8a7b650c574f8c8eab055053636370 P 1b732ab9a8764c11bc7dfab37c1e76cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e518880cdf24ad68dde3b4cdad50219 in term 0.
I20260811 17:19:11.968854 12592 raft_consensus.cc:2781] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:19:11.968940 12592 raft_consensus.cc:482] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:19:11.969010 12592 raft_consensus.cc:3037] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:11.970113 12592 raft_consensus.cc:504] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } }
I20260811 17:19:11.970534 12592 leader_election.cc:288] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219 [CANDIDATE]: Term 1 election: Requested vote from peers d8873a8068724601bb05db6d633a0ca6 (127.9.157.131:39833), 1b732ab9a8764c11bc7dfab37c1e76cc (127.9.157.130:40129)
I20260811 17:19:11.970870 12472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e8a7b650c574f8c8eab055053636370" candidate_uuid: "0e518880cdf24ad68dde3b4cdad50219" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8873a8068724601bb05db6d633a0ca6"
I20260811 17:19:11.970938 12213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e8a7b650c574f8c8eab055053636370" candidate_uuid: "0e518880cdf24ad68dde3b4cdad50219" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc"
I20260811 17:19:11.971019 12472 raft_consensus.cc:3037] T 1e8a7b650c574f8c8eab055053636370 P d8873a8068724601bb05db6d633a0ca6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:11.971050 12213 raft_consensus.cc:3037] T 1e8a7b650c574f8c8eab055053636370 P 1b732ab9a8764c11bc7dfab37c1e76cc [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:11.972555 12472 raft_consensus.cc:2454] T 1e8a7b650c574f8c8eab055053636370 P d8873a8068724601bb05db6d633a0ca6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e518880cdf24ad68dde3b4cdad50219 in term 1.
I20260811 17:19:11.972904 11867 leader_election.cc:302] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e518880cdf24ad68dde3b4cdad50219, d8873a8068724601bb05db6d633a0ca6; no voters:
I20260811 17:19:11.972982 12213 raft_consensus.cc:2454] T 1e8a7b650c574f8c8eab055053636370 P 1b732ab9a8764c11bc7dfab37c1e76cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e518880cdf24ad68dde3b4cdad50219 in term 1.
I20260811 17:19:11.973124 12592 raft_consensus.cc:2781] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:11.973404 12592 raft_consensus.cc:686] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219 [term 1 LEADER]: Becoming Leader. State: Replica: 0e518880cdf24ad68dde3b4cdad50219, State: Running, Role: LEADER
I20260811 17:19:11.973786 12592 consensus_queue.cc:237] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } }
I20260811 17:19:11.977159 11715 catalog_manager.cc:5495] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e518880cdf24ad68dde3b4cdad50219 (127.9.157.129). New cstate: current_term: 1 leader_uuid: "0e518880cdf24ad68dde3b4cdad50219" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } health_report { overall_health: UNKNOWN } } }
I20260811 17:19:12.011722 12575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:12.007768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40732 (local address 127.9.157.129:37597)
0811 17:19:12.007925 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:12.007929 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:12.007945 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:19:12.008118 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:12.008121 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:12.008179 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:12.008270 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:19:12.008526 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.009046 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.009985 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.010213 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.010307 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:12.010700 (+ 393us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:19:12.011209 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:19:12.011312 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:12.011429 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 17:19:12.011484 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 17:19:12.018637 12213 raft_consensus.cc:1261] T 1e8a7b650c574f8c8eab055053636370 P 1b732ab9a8764c11bc7dfab37c1e76cc [term 1 FOLLOWER]: Refusing update from remote peer 0e518880cdf24ad68dde3b4cdad50219: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:12.019558 12592 consensus_queue.cc:1035] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:12.020681 12472 raft_consensus.cc:1261] T 1e8a7b650c574f8c8eab055053636370 P d8873a8068724601bb05db6d633a0ca6 [term 1 FOLLOWER]: Refusing update from remote peer 0e518880cdf24ad68dde3b4cdad50219: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:12.021428 12592 consensus_queue.cc:1035] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:12.030151 12609 mvcc.cc:205] Tried to move back new op lower bound from 7317376008258830336 to 7317376008087699456. Current Snapshot: MvccSnapshot[applied={T|T < 7317376008258830336}]
I20260811 17:19:12.042858 11716 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:19:12.043092 11716 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:19:12.046033 11716 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 60b19882c4e5479fb29e3709af7fa42a: Sending DeleteTablet for 3 replicas of tablet 1e8a7b650c574f8c8eab055053636370
I20260811 17:19:12.047031 12436 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e8a7b650c574f8c8eab055053636370 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:12 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:48636
I20260811 17:19:12.047024 11946 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e8a7b650c574f8c8eab055053636370 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:12 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:40720
I20260811 17:19:12.047116 12184 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e8a7b650c574f8c8eab055053636370 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:12 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:36162
I20260811 17:19:12.047755 12620 tablet_replica.cc:330] T 1e8a7b650c574f8c8eab055053636370 P d8873a8068724601bb05db6d633a0ca6: stopping tablet replica
I20260811 17:19:12.047993 12619 tablet_replica.cc:330] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219: stopping tablet replica
I20260811 17:19:12.048367 12619 raft_consensus.cc:2229] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219 [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:19:12.048595 11716 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45062:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 17:19:12.048713 12620 raft_consensus.cc:2229] T 1e8a7b650c574f8c8eab055053636370 P d8873a8068724601bb05db6d633a0ca6 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 17:19:12.048969 11716 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:19:12.049345 12620 raft_consensus.cc:2258] T 1e8a7b650c574f8c8eab055053636370 P d8873a8068724601bb05db6d633a0ca6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:12.049880 12619 raft_consensus.cc:2258] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:12.050253 12623 tablet_replica.cc:330] T 1e8a7b650c574f8c8eab055053636370 P 1b732ab9a8764c11bc7dfab37c1e76cc: stopping tablet replica
I20260811 17:19:12.050556 12623 raft_consensus.cc:2229] T 1e8a7b650c574f8c8eab055053636370 P 1b732ab9a8764c11bc7dfab37c1e76cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:19:12.050871 12623 raft_consensus.cc:2258] T 1e8a7b650c574f8c8eab055053636370 P 1b732ab9a8764c11bc7dfab37c1e76cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:12.052091 12620 ts_tablet_manager.cc:1905] T 1e8a7b650c574f8c8eab055053636370 P d8873a8068724601bb05db6d633a0ca6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:12.052438 12623 ts_tablet_manager.cc:1905] T 1e8a7b650c574f8c8eab055053636370 P 1b732ab9a8764c11bc7dfab37c1e76cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:12.055008 12184 tablet_service.cc:1467] Processing CreateTablet for tablet 77ba69da3edf4dd3a55d59d2ff5839ec (DEFAULT_TABLE table=test-table [id=d4afd141e1dd49b3a90944c2eedb799a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:12.055154 12436 tablet_service.cc:1467] Processing CreateTablet for tablet 77ba69da3edf4dd3a55d59d2ff5839ec (DEFAULT_TABLE table=test-table [id=d4afd141e1dd49b3a90944c2eedb799a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:12.055261 12623 ts_tablet_manager.cc:1918] T 1e8a7b650c574f8c8eab055053636370 P 1b732ab9a8764c11bc7dfab37c1e76cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:19:12.055322 12184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77ba69da3edf4dd3a55d59d2ff5839ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:12.055359 12623 log.cc:1196] T 1e8a7b650c574f8c8eab055053636370 P 1b732ab9a8764c11bc7dfab37c1e76cc: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-1/wal/wals/1e8a7b650c574f8c8eab055053636370
I20260811 17:19:12.055348 12620 ts_tablet_manager.cc:1918] T 1e8a7b650c574f8c8eab055053636370 P d8873a8068724601bb05db6d633a0ca6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:19:12.055424 12436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77ba69da3edf4dd3a55d59d2ff5839ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:12.055716 12623 ts_tablet_manager.cc:1939] T 1e8a7b650c574f8c8eab055053636370 P 1b732ab9a8764c11bc7dfab37c1e76cc: Deleting consensus metadata
I20260811 17:19:12.056346 11946 tablet_service.cc:1467] Processing CreateTablet for tablet 77ba69da3edf4dd3a55d59d2ff5839ec (DEFAULT_TABLE table=test-table [id=d4afd141e1dd49b3a90944c2eedb799a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:12.056670 11946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77ba69da3edf4dd3a55d59d2ff5839ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:12.057085 11696 catalog_manager.cc:4841] TS 1b732ab9a8764c11bc7dfab37c1e76cc (127.9.157.130:40129): tablet 1e8a7b650c574f8c8eab055053636370 (table test-table [id=119d8dad43e845cb8bb70656e21171df]) successfully deleted
I20260811 17:19:12.058008 12591 tablet_bootstrap.cc:492] T 77ba69da3edf4dd3a55d59d2ff5839ec P d8873a8068724601bb05db6d633a0ca6: Bootstrap starting.
I20260811 17:19:12.058701 12585 tablet_bootstrap.cc:492] T 77ba69da3edf4dd3a55d59d2ff5839ec P 1b732ab9a8764c11bc7dfab37c1e76cc: Bootstrap starting.
I20260811 17:19:12.059125 12591 tablet_bootstrap.cc:654] T 77ba69da3edf4dd3a55d59d2ff5839ec P d8873a8068724601bb05db6d633a0ca6: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:12.059679 12585 tablet_bootstrap.cc:654] T 77ba69da3edf4dd3a55d59d2ff5839ec P 1b732ab9a8764c11bc7dfab37c1e76cc: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:12.060043 12583 tablet_bootstrap.cc:492] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219: Bootstrap starting.
I20260811 17:19:12.055438 12620 log.cc:1196] T 1e8a7b650c574f8c8eab055053636370 P d8873a8068724601bb05db6d633a0ca6: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-2/wal/wals/1e8a7b650c574f8c8eab055053636370
I20260811 17:19:12.061115 12583 tablet_bootstrap.cc:654] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:12.061177 12591 tablet_bootstrap.cc:492] T 77ba69da3edf4dd3a55d59d2ff5839ec P d8873a8068724601bb05db6d633a0ca6: No bootstrap required, opened a new log
I20260811 17:19:12.061254 12591 ts_tablet_manager.cc:1397] T 77ba69da3edf4dd3a55d59d2ff5839ec P d8873a8068724601bb05db6d633a0ca6: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 17:19:12.061317 12620 ts_tablet_manager.cc:1939] T 1e8a7b650c574f8c8eab055053636370 P d8873a8068724601bb05db6d633a0ca6: Deleting consensus metadata
I20260811 17:19:12.061684 12591 raft_consensus.cc:348] T 77ba69da3edf4dd3a55d59d2ff5839ec P d8873a8068724601bb05db6d633a0ca6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } }
I20260811 17:19:12.061785 12591 raft_consensus.cc:374] T 77ba69da3edf4dd3a55d59d2ff5839ec P d8873a8068724601bb05db6d633a0ca6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:12.061813 12591 raft_consensus.cc:729] T 77ba69da3edf4dd3a55d59d2ff5839ec P d8873a8068724601bb05db6d633a0ca6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8873a8068724601bb05db6d633a0ca6, State: Initialized, Role: FOLLOWER
I20260811 17:19:12.061932 12591 consensus_queue.cc:260] T 77ba69da3edf4dd3a55d59d2ff5839ec P d8873a8068724601bb05db6d633a0ca6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } }
I20260811 17:19:12.062639 11694 catalog_manager.cc:4841] TS d8873a8068724601bb05db6d633a0ca6 (127.9.157.131:39833): tablet 1e8a7b650c574f8c8eab055053636370 (table test-table [id=119d8dad43e845cb8bb70656e21171df]) successfully deleted
I20260811 17:19:12.064117 12585 tablet_bootstrap.cc:492] T 77ba69da3edf4dd3a55d59d2ff5839ec P 1b732ab9a8764c11bc7dfab37c1e76cc: No bootstrap required, opened a new log
I20260811 17:19:12.064116 12619 ts_tablet_manager.cc:1905] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:12.064211 12585 ts_tablet_manager.cc:1397] T 77ba69da3edf4dd3a55d59d2ff5839ec P 1b732ab9a8764c11bc7dfab37c1e76cc: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260811 17:19:12.064689 12585 raft_consensus.cc:348] T 77ba69da3edf4dd3a55d59d2ff5839ec P 1b732ab9a8764c11bc7dfab37c1e76cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } }
I20260811 17:19:12.064807 12585 raft_consensus.cc:374] T 77ba69da3edf4dd3a55d59d2ff5839ec P 1b732ab9a8764c11bc7dfab37c1e76cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:12.064836 12585 raft_consensus.cc:729] T 77ba69da3edf4dd3a55d59d2ff5839ec P 1b732ab9a8764c11bc7dfab37c1e76cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b732ab9a8764c11bc7dfab37c1e76cc, State: Initialized, Role: FOLLOWER
I20260811 17:19:12.064954 12585 consensus_queue.cc:260] T 77ba69da3edf4dd3a55d59d2ff5839ec P 1b732ab9a8764c11bc7dfab37c1e76cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } }
I20260811 17:19:12.065192 12585 ts_tablet_manager.cc:1428] T 77ba69da3edf4dd3a55d59d2ff5839ec P 1b732ab9a8764c11bc7dfab37c1e76cc: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 17:19:12.066147 12591 ts_tablet_manager.cc:1428] T 77ba69da3edf4dd3a55d59d2ff5839ec P d8873a8068724601bb05db6d633a0ca6: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260811 17:19:12.067166 12619 ts_tablet_manager.cc:1918] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:19:12.067271 12619 log.cc:1196] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-0/wal/wals/1e8a7b650c574f8c8eab055053636370
I20260811 17:19:12.067628 12619 ts_tablet_manager.cc:1939] T 1e8a7b650c574f8c8eab055053636370 P 0e518880cdf24ad68dde3b4cdad50219: Deleting consensus metadata
I20260811 17:19:12.069254 12583 tablet_bootstrap.cc:492] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219: No bootstrap required, opened a new log
I20260811 17:19:12.069348 12583 ts_tablet_manager.cc:1397] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260811 17:19:12.069440 11692 catalog_manager.cc:4841] TS 0e518880cdf24ad68dde3b4cdad50219 (127.9.157.129:37597): tablet 1e8a7b650c574f8c8eab055053636370 (table test-table [id=119d8dad43e845cb8bb70656e21171df]) successfully deleted
I20260811 17:19:12.069842 12583 raft_consensus.cc:348] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } }
I20260811 17:19:12.069972 12583 raft_consensus.cc:374] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:12.070012 12583 raft_consensus.cc:729] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e518880cdf24ad68dde3b4cdad50219, State: Initialized, Role: FOLLOWER
I20260811 17:19:12.070178 12583 consensus_queue.cc:260] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } }
I20260811 17:19:12.070458 12583 ts_tablet_manager.cc:1428] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20260811 17:19:12.102942 12546 tablet.cc:2369] T 77ba69da3edf4dd3a55d59d2ff5839ec P d8873a8068724601bb05db6d633a0ca6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:19:12.121523 12309 tablet.cc:2369] T 77ba69da3edf4dd3a55d59d2ff5839ec P 1b732ab9a8764c11bc7dfab37c1e76cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:19:12.132225 12055 tablet.cc:2369] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:19:12.208830 12602 raft_consensus.cc:482] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:19:12.209004 12602 raft_consensus.cc:504] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } }
I20260811 17:19:12.209575 12602 leader_election.cc:288] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b732ab9a8764c11bc7dfab37c1e76cc (127.9.157.130:40129), d8873a8068724601bb05db6d633a0ca6 (127.9.157.131:39833)
I20260811 17:19:12.210143 12213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77ba69da3edf4dd3a55d59d2ff5839ec" candidate_uuid: "0e518880cdf24ad68dde3b4cdad50219" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" is_pre_election: true
I20260811 17:19:12.210146 12472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77ba69da3edf4dd3a55d59d2ff5839ec" candidate_uuid: "0e518880cdf24ad68dde3b4cdad50219" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8873a8068724601bb05db6d633a0ca6" is_pre_election: true
I20260811 17:19:12.210332 12472 raft_consensus.cc:2454] T 77ba69da3edf4dd3a55d59d2ff5839ec P d8873a8068724601bb05db6d633a0ca6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e518880cdf24ad68dde3b4cdad50219 in term 0.
I20260811 17:19:12.210337 12213 raft_consensus.cc:2454] T 77ba69da3edf4dd3a55d59d2ff5839ec P 1b732ab9a8764c11bc7dfab37c1e76cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e518880cdf24ad68dde3b4cdad50219 in term 0.
I20260811 17:19:12.210692 11867 leader_election.cc:302] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e518880cdf24ad68dde3b4cdad50219, d8873a8068724601bb05db6d633a0ca6; no voters:
I20260811 17:19:12.210901 12602 raft_consensus.cc:2781] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:19:12.210990 12602 raft_consensus.cc:482] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:19:12.211047 12602 raft_consensus.cc:3037] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:12.212134 12602 raft_consensus.cc:504] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } }
I20260811 17:19:12.212533 12602 leader_election.cc:288] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219 [CANDIDATE]: Term 1 election: Requested vote from peers 1b732ab9a8764c11bc7dfab37c1e76cc (127.9.157.130:40129), d8873a8068724601bb05db6d633a0ca6 (127.9.157.131:39833)
I20260811 17:19:12.212950 12472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77ba69da3edf4dd3a55d59d2ff5839ec" candidate_uuid: "0e518880cdf24ad68dde3b4cdad50219" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8873a8068724601bb05db6d633a0ca6"
I20260811 17:19:12.212960 12213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77ba69da3edf4dd3a55d59d2ff5839ec" candidate_uuid: "0e518880cdf24ad68dde3b4cdad50219" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc"
I20260811 17:19:12.213075 12213 raft_consensus.cc:3037] T 77ba69da3edf4dd3a55d59d2ff5839ec P 1b732ab9a8764c11bc7dfab37c1e76cc [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:12.213106 12472 raft_consensus.cc:3037] T 77ba69da3edf4dd3a55d59d2ff5839ec P d8873a8068724601bb05db6d633a0ca6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:12.214075 12472 raft_consensus.cc:2454] T 77ba69da3edf4dd3a55d59d2ff5839ec P d8873a8068724601bb05db6d633a0ca6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e518880cdf24ad68dde3b4cdad50219 in term 1.
I20260811 17:19:12.214319 12213 raft_consensus.cc:2454] T 77ba69da3edf4dd3a55d59d2ff5839ec P 1b732ab9a8764c11bc7dfab37c1e76cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e518880cdf24ad68dde3b4cdad50219 in term 1.
I20260811 17:19:12.214416 11867 leader_election.cc:302] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e518880cdf24ad68dde3b4cdad50219, d8873a8068724601bb05db6d633a0ca6; no voters:
I20260811 17:19:12.214655 12602 raft_consensus.cc:2781] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:12.214764 12602 raft_consensus.cc:686] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219 [term 1 LEADER]: Becoming Leader. State: Replica: 0e518880cdf24ad68dde3b4cdad50219, State: Running, Role: LEADER
I20260811 17:19:12.214920 12602 consensus_queue.cc:237] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } }
I20260811 17:19:12.216723 11716 catalog_manager.cc:5495] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e518880cdf24ad68dde3b4cdad50219 (127.9.157.129). New cstate: current_term: 1 leader_uuid: "0e518880cdf24ad68dde3b4cdad50219" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } health_report { overall_health: UNKNOWN } } }
Aug 11 17:19:12 dist-test-slave-mjjr krb5kdc[11631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468750, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:12.250506 11809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:12.241731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45078 (local address 127.9.157.190:41783)
0811 17:19:12.241855 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:12.241859 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:12.242126 (+ 267us) server_negotiation.cc:407] Connection header received
0811 17:19:12.242293 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:12.242296 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:12.242341 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:12.242435 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:12.243141 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.243751 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.244549 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.244765 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.244876 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:12.247259 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:12.247262 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:12.247264 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:12.247285 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:12.248629 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:12.249121 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:12.249131 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:12.249133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:12.249200 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:12.249494 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:12.249499 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:12.249500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:12.249694 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:12.249810 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:12.250142 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 17:19:12.250352 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":140,"server-negotiator.queue_time_us":41}
I20260811 17:19:12.250507 12640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:12.241697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:41783 (local address 127.0.0.1:45078)
0811 17:19:12.242034 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:12.242047 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:12.242190 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:12.242458 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:12.242460 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:12.242920 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:12.243019 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:12.243025 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.243874 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.243877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:12.244431 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:12.244439 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.244883 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.244886 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:12.245007 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:12.245566 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:12.245583 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:12.247100 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:12.248752 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:12.248762 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:12.248770 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:12.249008 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:12.249305 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:12.249308 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:12.249310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:12.249402 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:12.249812 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:12.249823 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:12.249965 (+ 142us) client_negotiation.cc:769] Sending connection context
0811 17:19:12.250090 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 17:19:12.250289 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":1,"thread_start_us":135,"threads_started":1}
I20260811 17:19:12.255975 11716 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:45078:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 17:19:12.257016 11716 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:19:12.262871 11946 tablet_service.cc:1467] Processing CreateTablet for tablet de344aca2cc143dfbd0d000c979ab593 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fb55c5398bab45d697c57e65d293ba9e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 17:19:12.262902 12436 tablet_service.cc:1467] Processing CreateTablet for tablet de344aca2cc143dfbd0d000c979ab593 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fb55c5398bab45d697c57e65d293ba9e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 17:19:12.263429 12436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de344aca2cc143dfbd0d000c979ab593. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:12.263581 11946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de344aca2cc143dfbd0d000c979ab593. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:12.262835 12184 tablet_service.cc:1467] Processing CreateTablet for tablet de344aca2cc143dfbd0d000c979ab593 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fb55c5398bab45d697c57e65d293ba9e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 17:19:12.265662 12184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de344aca2cc143dfbd0d000c979ab593. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:12.266259 12583 tablet_bootstrap.cc:492] T de344aca2cc143dfbd0d000c979ab593 P 0e518880cdf24ad68dde3b4cdad50219: Bootstrap starting.
I20260811 17:19:12.266342 12591 tablet_bootstrap.cc:492] T de344aca2cc143dfbd0d000c979ab593 P d8873a8068724601bb05db6d633a0ca6: Bootstrap starting.
I20260811 17:19:12.267160 12583 tablet_bootstrap.cc:654] T de344aca2cc143dfbd0d000c979ab593 P 0e518880cdf24ad68dde3b4cdad50219: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:12.267190 12591 tablet_bootstrap.cc:654] T de344aca2cc143dfbd0d000c979ab593 P d8873a8068724601bb05db6d633a0ca6: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:12.268602 12583 tablet_bootstrap.cc:492] T de344aca2cc143dfbd0d000c979ab593 P 0e518880cdf24ad68dde3b4cdad50219: No bootstrap required, opened a new log
I20260811 17:19:12.268692 12583 ts_tablet_manager.cc:1397] T de344aca2cc143dfbd0d000c979ab593 P 0e518880cdf24ad68dde3b4cdad50219: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 17:19:12.268935 12585 tablet_bootstrap.cc:492] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc: Bootstrap starting.
I20260811 17:19:12.268983 12591 tablet_bootstrap.cc:492] T de344aca2cc143dfbd0d000c979ab593 P d8873a8068724601bb05db6d633a0ca6: No bootstrap required, opened a new log
I20260811 17:19:12.269070 12591 ts_tablet_manager.cc:1397] T de344aca2cc143dfbd0d000c979ab593 P d8873a8068724601bb05db6d633a0ca6: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 17:19:12.269137 12583 raft_consensus.cc:348] T de344aca2cc143dfbd0d000c979ab593 P 0e518880cdf24ad68dde3b4cdad50219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } }
I20260811 17:19:12.269243 12583 raft_consensus.cc:374] T de344aca2cc143dfbd0d000c979ab593 P 0e518880cdf24ad68dde3b4cdad50219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:12.269275 12583 raft_consensus.cc:729] T de344aca2cc143dfbd0d000c979ab593 P 0e518880cdf24ad68dde3b4cdad50219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e518880cdf24ad68dde3b4cdad50219, State: Initialized, Role: FOLLOWER
I20260811 17:19:12.269418 12583 consensus_queue.cc:260] T de344aca2cc143dfbd0d000c979ab593 P 0e518880cdf24ad68dde3b4cdad50219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } }
I20260811 17:19:12.269469 12591 raft_consensus.cc:348] T de344aca2cc143dfbd0d000c979ab593 P d8873a8068724601bb05db6d633a0ca6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } }
I20260811 17:19:12.269618 12591 raft_consensus.cc:374] T de344aca2cc143dfbd0d000c979ab593 P d8873a8068724601bb05db6d633a0ca6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:12.269654 12591 raft_consensus.cc:729] T de344aca2cc143dfbd0d000c979ab593 P d8873a8068724601bb05db6d633a0ca6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8873a8068724601bb05db6d633a0ca6, State: Initialized, Role: FOLLOWER
I20260811 17:19:12.269681 12583 ts_tablet_manager.cc:1428] T de344aca2cc143dfbd0d000c979ab593 P 0e518880cdf24ad68dde3b4cdad50219: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 17:19:12.269742 12602 tablet_replica.cc:439] T de344aca2cc143dfbd0d000c979ab593 P 0e518880cdf24ad68dde3b4cdad50219: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } } }
I20260811 17:19:12.269802 12591 consensus_queue.cc:260] T de344aca2cc143dfbd0d000c979ab593 P d8873a8068724601bb05db6d633a0ca6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } }
I20260811 17:19:12.269886 12602 tablet_replica.cc:442] T de344aca2cc143dfbd0d000c979ab593 P 0e518880cdf24ad68dde3b4cdad50219: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:19:12.269912 12585 tablet_bootstrap.cc:654] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:12.270078 12591 ts_tablet_manager.cc:1428] T de344aca2cc143dfbd0d000c979ab593 P d8873a8068724601bb05db6d633a0ca6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 17:19:12.270092 12595 tablet_replica.cc:439] T de344aca2cc143dfbd0d000c979ab593 P d8873a8068724601bb05db6d633a0ca6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } } }
I20260811 17:19:12.270792 12595 tablet_replica.cc:442] T de344aca2cc143dfbd0d000c979ab593 P d8873a8068724601bb05db6d633a0ca6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:19:12.271452 12585 tablet_bootstrap.cc:492] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc: No bootstrap required, opened a new log
I20260811 17:19:12.271564 12585 ts_tablet_manager.cc:1397] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260811 17:19:12.272085 12585 raft_consensus.cc:348] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } }
I20260811 17:19:12.272209 12585 raft_consensus.cc:374] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:12.272248 12585 raft_consensus.cc:729] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b732ab9a8764c11bc7dfab37c1e76cc, State: Initialized, Role: FOLLOWER
I20260811 17:19:12.272392 12585 consensus_queue.cc:260] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } }
I20260811 17:19:12.272697 12585 ts_tablet_manager.cc:1428] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 17:19:12.272644 12588 tablet_replica.cc:439] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } } }
I20260811 17:19:12.272809 12588 tablet_replica.cc:442] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:19:12.484094 12588 raft_consensus.cc:482] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:19:12.484313 12588 raft_consensus.cc:504] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } }
I20260811 17:19:12.485747 12588 leader_election.cc:288] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e518880cdf24ad68dde3b4cdad50219 (127.9.157.129:37597), d8873a8068724601bb05db6d633a0ca6 (127.9.157.131:39833)
I20260811 17:19:12.491555 12575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:12.486067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:52395 (local address 127.9.157.129:37597)
0811 17:19:12.486209 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:12.486239 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:12.486475 (+ 236us) server_negotiation.cc:407] Connection header received
0811 17:19:12.486634 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:12.486638 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:12.486698 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:12.486791 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:12.487324 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.488189 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.490102 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.490710 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.490864 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:12.490920 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:12.491332 (+ 412us) server_negotiation.cc:299] Negotiation successful
0811 17:19:12.491410 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 17:19:12.491557 12650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:12.486155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:49113 (local address 127.9.157.131:39833)
0811 17:19:12.486448 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:12.486455 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:12.486527 (+ 72us) server_negotiation.cc:407] Connection header received
0811 17:19:12.486638 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:12.486641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:12.486698 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:12.486792 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:12.487324 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.488189 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.490102 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.490741 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.490859 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:12.490920 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:12.491332 (+ 412us) server_negotiation.cc:299] Negotiation successful
0811 17:19:12.491412 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":65,"threads_started":1}
I20260811 17:19:12.491621 12646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:12.485984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:37597 (local address 127.9.157.130:52395)
0811 17:19:12.486342 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:12.486359 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:12.486511 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:12.486822 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:12.486826 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:12.486870 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:12.487186 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:12.487192 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.488332 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.488337 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:12.489960 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:12.489974 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.490862 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.490866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:12.491107 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:12.491215 (+ 108us) client_negotiation.cc:769] Sending connection context
0811 17:19:12.491299 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 17:19:12.491423 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":367,"spinlock_wait_cycles":31232,"thread_start_us":91,"threads_started":1}
I20260811 17:19:12.491703 12647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:12.486078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:39833 (local address 127.9.157.130:49113)
0811 17:19:12.486432 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:12.486459 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:12.486538 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:12.486822 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:12.486826 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:12.486871 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:12.487183 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:12.487192 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.488332 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.488337 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:12.489960 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:12.489974 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.490862 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.490866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:12.491109 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:12.491213 (+ 104us) client_negotiation.cc:769] Sending connection context
0811 17:19:12.491299 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 17:19:12.491423 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":350,"spinlock_wait_cycles":17920,"thread_start_us":82,"threads_started":1}
I20260811 17:19:12.492355 11975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de344aca2cc143dfbd0d000c979ab593" candidate_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e518880cdf24ad68dde3b4cdad50219" is_pre_election: true
I20260811 17:19:12.492574 12472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de344aca2cc143dfbd0d000c979ab593" candidate_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8873a8068724601bb05db6d633a0ca6" is_pre_election: true
I20260811 17:19:12.492695 11975 raft_consensus.cc:2454] T de344aca2cc143dfbd0d000c979ab593 P 0e518880cdf24ad68dde3b4cdad50219 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b732ab9a8764c11bc7dfab37c1e76cc in term 0.
I20260811 17:19:12.492735 12472 raft_consensus.cc:2454] T de344aca2cc143dfbd0d000c979ab593 P d8873a8068724601bb05db6d633a0ca6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b732ab9a8764c11bc7dfab37c1e76cc in term 0.
I20260811 17:19:12.493155 12107 leader_election.cc:302] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b732ab9a8764c11bc7dfab37c1e76cc, d8873a8068724601bb05db6d633a0ca6; no voters:
I20260811 17:19:12.493474 12588 raft_consensus.cc:2781] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:19:12.493536 12588 raft_consensus.cc:482] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:19:12.493564 12588 raft_consensus.cc:3037] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:12.494709 12588 raft_consensus.cc:504] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } }
I20260811 17:19:12.495134 12588 leader_election.cc:288] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc [CANDIDATE]: Term 1 election: Requested vote from peers 0e518880cdf24ad68dde3b4cdad50219 (127.9.157.129:37597), d8873a8068724601bb05db6d633a0ca6 (127.9.157.131:39833)
I20260811 17:19:12.495484 11975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de344aca2cc143dfbd0d000c979ab593" candidate_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e518880cdf24ad68dde3b4cdad50219"
I20260811 17:19:12.495594 12472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de344aca2cc143dfbd0d000c979ab593" candidate_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8873a8068724601bb05db6d633a0ca6"
I20260811 17:19:12.495613 11975 raft_consensus.cc:3037] T de344aca2cc143dfbd0d000c979ab593 P 0e518880cdf24ad68dde3b4cdad50219 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:12.495761 12472 raft_consensus.cc:3037] T de344aca2cc143dfbd0d000c979ab593 P d8873a8068724601bb05db6d633a0ca6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:12.496719 12472 raft_consensus.cc:2454] T de344aca2cc143dfbd0d000c979ab593 P d8873a8068724601bb05db6d633a0ca6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b732ab9a8764c11bc7dfab37c1e76cc in term 1.
I20260811 17:19:12.496726 11975 raft_consensus.cc:2454] T de344aca2cc143dfbd0d000c979ab593 P 0e518880cdf24ad68dde3b4cdad50219 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b732ab9a8764c11bc7dfab37c1e76cc in term 1.
I20260811 17:19:12.497048 12107 leader_election.cc:302] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b732ab9a8764c11bc7dfab37c1e76cc, d8873a8068724601bb05db6d633a0ca6; no voters:
I20260811 17:19:12.497243 12588 raft_consensus.cc:2781] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:12.497502 12588 raft_consensus.cc:686] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc [term 1 LEADER]: Becoming Leader. State: Replica: 1b732ab9a8764c11bc7dfab37c1e76cc, State: Running, Role: LEADER
I20260811 17:19:12.497831 12588 consensus_queue.cc:237] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } }
I20260811 17:19:12.499390 12652 tablet_replica.cc:439] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc: TxnStatusTablet state changed. Reason: New leader 1b732ab9a8764c11bc7dfab37c1e76cc. Latest consensus state: current_term: 1 leader_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } } }
I20260811 17:19:12.499514 12652 tablet_replica.cc:442] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc: This TxnStatusTablet replica's current role is: LEADER
I20260811 17:19:12.499897 12657 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 17:19:12.500114 12657 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 17:19:12.500511 11713 catalog_manager.cc:5495] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b732ab9a8764c11bc7dfab37c1e76cc (127.9.157.130). New cstate: current_term: 1 leader_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } health_report { overall_health: UNKNOWN } } }
I20260811 17:19:12.565704 12640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:12.561479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:40129 (local address 127.0.0.1:36164)
0811 17:19:12.561644 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:12.561663 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:12.561805 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:12.562202 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:12.562205 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:12.562213 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 17:19:12.562343 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:12.562350 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.563161 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.563165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:12.563862 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:12.563892 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.564457 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.564461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:12.564560 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:12.564611 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 17:19:12.565361 (+ 750us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 17:19:12.565381 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 17:19:12.565473 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 17:19:12.565573 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 17:19:12.565723 12660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:12.561623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36164 (local address 127.9.157.130:40129)
0811 17:19:12.561917 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:12.561924 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:12.561946 (+ 22us) server_negotiation.cc:407] Connection header received
0811 17:19:12.562015 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:12.562017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:12.562071 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:12.562189 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:19:12.562460 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.563038 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.564040 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.564333 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.564454 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:12.564786 (+ 332us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:19:12.565251 (+ 465us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:19:12.565383 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:12.565485 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 17:19:12.565545 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":99,"threads_started":1}
I20260811 17:19:12.569798 12472 raft_consensus.cc:1261] T de344aca2cc143dfbd0d000c979ab593 P d8873a8068724601bb05db6d633a0ca6 [term 1 FOLLOWER]: Refusing update from remote peer 1b732ab9a8764c11bc7dfab37c1e76cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:12.569967 11975 raft_consensus.cc:1261] T de344aca2cc143dfbd0d000c979ab593 P 0e518880cdf24ad68dde3b4cdad50219 [term 1 FOLLOWER]: Refusing update from remote peer 1b732ab9a8764c11bc7dfab37c1e76cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:12.570569 12652 consensus_queue.cc:1035] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:12.570782 12588 consensus_queue.cc:1035] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:12.572521 12595 tablet_replica.cc:439] T de344aca2cc143dfbd0d000c979ab593 P d8873a8068724601bb05db6d633a0ca6: TxnStatusTablet state changed. Reason: New leader 1b732ab9a8764c11bc7dfab37c1e76cc. Latest consensus state: current_term: 1 leader_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } } }
I20260811 17:19:12.572654 12595 tablet_replica.cc:442] T de344aca2cc143dfbd0d000c979ab593 P d8873a8068724601bb05db6d633a0ca6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:19:12.572752 12602 tablet_replica.cc:439] T de344aca2cc143dfbd0d000c979ab593 P 0e518880cdf24ad68dde3b4cdad50219: TxnStatusTablet state changed. Reason: New leader 1b732ab9a8764c11bc7dfab37c1e76cc. Latest consensus state: current_term: 1 leader_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } } }
I20260811 17:19:12.572882 12602 tablet_replica.cc:442] T de344aca2cc143dfbd0d000c979ab593 P 0e518880cdf24ad68dde3b4cdad50219: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:19:12.576004 12661 mvcc.cc:205] Tried to move back new op lower bound from 7317376010519732224 to 7317376010233700352. Current Snapshot: MvccSnapshot[applied={T|T < 7317376010519732224}]
I20260811 17:19:12.576164 12652 tablet_replica.cc:439] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } } }
I20260811 17:19:12.576313 12652 tablet_replica.cc:442] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc: This TxnStatusTablet replica's current role is: LEADER
I20260811 17:19:12.576474 12652 tablet_replica.cc:439] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } } }
I20260811 17:19:12.576581 12652 tablet_replica.cc:442] T de344aca2cc143dfbd0d000c979ab593 P 1b732ab9a8764c11bc7dfab37c1e76cc: This TxnStatusTablet replica's current role is: LEADER
I20260811 17:19:12.576571 12595 tablet_replica.cc:439] T de344aca2cc143dfbd0d000c979ab593 P d8873a8068724601bb05db6d633a0ca6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } } }
I20260811 17:19:12.576697 12595 tablet_replica.cc:442] T de344aca2cc143dfbd0d000c979ab593 P d8873a8068724601bb05db6d633a0ca6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:19:12.577168 12663 mvcc.cc:205] Tried to move back new op lower bound from 7317376010519732224 to 7317376010233700352. Current Snapshot: MvccSnapshot[applied={T|T < 7317376010519732224}]
I20260811 17:19:12.577188 12602 tablet_replica.cc:439] T de344aca2cc143dfbd0d000c979ab593 P 0e518880cdf24ad68dde3b4cdad50219: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e518880cdf24ad68dde3b4cdad50219" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 37597 } } peers { permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 } } peers { permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 } } }
I20260811 17:19:12.577440 12602 tablet_replica.cc:442] T de344aca2cc143dfbd0d000c979ab593 P 0e518880cdf24ad68dde3b4cdad50219: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:19:12.599711 12660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:12.595387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:40357 (local address 127.9.157.130:40129)
0811 17:19:12.595706 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:12.595715 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:12.595882 (+ 167us) server_negotiation.cc:407] Connection header received
0811 17:19:12.596153 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:12.596160 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:12.596213 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:12.596287 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:19:12.596589 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.597129 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.598374 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.598599 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.598694 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:12.599079 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:19:12.599214 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:19:12.599311 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:12.599450 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 17:19:12.599536 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
I20260811 17:19:12.599720 12674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:12.595293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:40129 (local address 127.9.157.129:40357)
0811 17:19:12.595712 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:12.595766 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:12.595897 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:12.596331 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:12.596334 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:12.596344 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 17:19:12.596465 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:12.596470 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.597248 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.597252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:12.598202 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:12.598216 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.598740 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.598744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:12.598834 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:12.598899 (+ 65us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 17:19:12.599331 (+ 432us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 17:19:12.599339 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 17:19:12.599418 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 17:19:12.599516 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":148,"threads_started":1}
I20260811 17:19:12.608634 12213 raft_consensus.cc:1261] T 77ba69da3edf4dd3a55d59d2ff5839ec P 1b732ab9a8764c11bc7dfab37c1e76cc [term 1 FOLLOWER]: Refusing update from remote peer 0e518880cdf24ad68dde3b4cdad50219: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:12.609108 12472 raft_consensus.cc:1261] T 77ba69da3edf4dd3a55d59d2ff5839ec P d8873a8068724601bb05db6d633a0ca6 [term 1 FOLLOWER]: Refusing update from remote peer 0e518880cdf24ad68dde3b4cdad50219: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:12.609117 12602 consensus_queue.cc:1035] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b732ab9a8764c11bc7dfab37c1e76cc" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:12.609563 12602 consensus_queue.cc:1035] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8873a8068724601bb05db6d633a0ca6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:12.643559 12575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:12.638320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:42815 (local address 127.9.157.129:37597)
0811 17:19:12.638430 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:12.638435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:12.638628 (+ 193us) server_negotiation.cc:407] Connection header received
0811 17:19:12.638871 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:12.638875 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:12.638937 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:12.638992 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:19:12.639879 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.640631 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.641986 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.642345 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.642452 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:12.643003 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:19:12.643106 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:19:12.643161 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:12.643346 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 17:19:12.643414 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 17:19:12.644414 12679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:12.637910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:37597 (local address 127.9.157.130:42815)
0811 17:19:12.638534 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:12.638552 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:12.638718 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:12.639295 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:12.639298 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:12.639308 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 17:19:12.639724 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:12.639733 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.640877 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.640881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:12.641825 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:12.641837 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.642650 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.642654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:12.642745 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:12.642808 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 17:19:12.643235 (+ 427us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 17:19:12.643241 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 17:19:12.644148 (+ 907us) client_negotiation.cc:240] Negotiation successful
0811 17:19:12.644241 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":153,"threads_started":1}
I20260811 17:19:12.674014 11713 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:19:12.674204 11713 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:19:12.676486 11713 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 60b19882c4e5479fb29e3709af7fa42a: Sending DeleteTablet for 3 replicas of tablet 77ba69da3edf4dd3a55d59d2ff5839ec
I20260811 17:19:12.677119 11946 tablet_service.cc:1514] Processing DeleteTablet for tablet 77ba69da3edf4dd3a55d59d2ff5839ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:12 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:40720
I20260811 17:19:12.677211 12184 tablet_service.cc:1514] Processing DeleteTablet for tablet 77ba69da3edf4dd3a55d59d2ff5839ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:12 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:36162
I20260811 17:19:12.677413 12436 tablet_service.cc:1514] Processing DeleteTablet for tablet 77ba69da3edf4dd3a55d59d2ff5839ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:12 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:48636
I20260811 17:19:12.677582 12681 tablet_replica.cc:330] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219: stopping tablet replica
I20260811 17:19:12.677639 12682 tablet_replica.cc:330] T 77ba69da3edf4dd3a55d59d2ff5839ec P 1b732ab9a8764c11bc7dfab37c1e76cc: stopping tablet replica
I20260811 17:19:12.677781 12682 raft_consensus.cc:2229] T 77ba69da3edf4dd3a55d59d2ff5839ec P 1b732ab9a8764c11bc7dfab37c1e76cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:19:12.677793 12681 raft_consensus.cc:2229] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219 [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:19:12.677953 12682 raft_consensus.cc:2258] T 77ba69da3edf4dd3a55d59d2ff5839ec P 1b732ab9a8764c11bc7dfab37c1e76cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:12.678041 12681 raft_consensus.cc:2258] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:12.678287 12684 tablet_replica.cc:330] T 77ba69da3edf4dd3a55d59d2ff5839ec P d8873a8068724601bb05db6d633a0ca6: stopping tablet replica
I20260811 17:19:12.678500 12684 raft_consensus.cc:2229] T 77ba69da3edf4dd3a55d59d2ff5839ec P d8873a8068724601bb05db6d633a0ca6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:19:12.678691 12684 raft_consensus.cc:2258] T 77ba69da3edf4dd3a55d59d2ff5839ec P d8873a8068724601bb05db6d633a0ca6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:12.679163 12681 ts_tablet_manager.cc:1905] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:12.679617 12684 ts_tablet_manager.cc:1905] T 77ba69da3edf4dd3a55d59d2ff5839ec P d8873a8068724601bb05db6d633a0ca6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:12.679677 12682 ts_tablet_manager.cc:1905] T 77ba69da3edf4dd3a55d59d2ff5839ec P 1b732ab9a8764c11bc7dfab37c1e76cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:12.682636 12682 ts_tablet_manager.cc:1918] T 77ba69da3edf4dd3a55d59d2ff5839ec P 1b732ab9a8764c11bc7dfab37c1e76cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:19:12.682675 12681 ts_tablet_manager.cc:1918] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:19:12.682734 12682 log.cc:1196] T 77ba69da3edf4dd3a55d59d2ff5839ec P 1b732ab9a8764c11bc7dfab37c1e76cc: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-1/wal/wals/77ba69da3edf4dd3a55d59d2ff5839ec
I20260811 17:19:12.682756 12681 log.cc:1196] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-0/wal/wals/77ba69da3edf4dd3a55d59d2ff5839ec
I20260811 17:19:12.682896 12684 ts_tablet_manager.cc:1918] T 77ba69da3edf4dd3a55d59d2ff5839ec P d8873a8068724601bb05db6d633a0ca6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:19:12.683018 12684 log.cc:1196] T 77ba69da3edf4dd3a55d59d2ff5839ec P d8873a8068724601bb05db6d633a0ca6: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786468743878666-9846-0/minicluster-data/ts-2/wal/wals/77ba69da3edf4dd3a55d59d2ff5839ec
I20260811 17:19:12.683104 12681 ts_tablet_manager.cc:1939] T 77ba69da3edf4dd3a55d59d2ff5839ec P 0e518880cdf24ad68dde3b4cdad50219: Deleting consensus metadata
I20260811 17:19:12.683079 12682 ts_tablet_manager.cc:1939] T 77ba69da3edf4dd3a55d59d2ff5839ec P 1b732ab9a8764c11bc7dfab37c1e76cc: Deleting consensus metadata
I20260811 17:19:12.683444 12684 ts_tablet_manager.cc:1939] T 77ba69da3edf4dd3a55d59d2ff5839ec P d8873a8068724601bb05db6d633a0ca6: Deleting consensus metadata
I20260811 17:19:12.684002 11692 catalog_manager.cc:4841] TS 0e518880cdf24ad68dde3b4cdad50219 (127.9.157.129:37597): tablet 77ba69da3edf4dd3a55d59d2ff5839ec (table test-table [id=d4afd141e1dd49b3a90944c2eedb799a]) successfully deleted
I20260811 17:19:12.684129 11696 catalog_manager.cc:4841] TS 1b732ab9a8764c11bc7dfab37c1e76cc (127.9.157.130:40129): tablet 77ba69da3edf4dd3a55d59d2ff5839ec (table test-table [id=d4afd141e1dd49b3a90944c2eedb799a]) successfully deleted
I20260811 17:19:12.684240 11694 catalog_manager.cc:4841] TS d8873a8068724601bb05db6d633a0ca6 (127.9.157.131:39833): tablet 77ba69da3edf4dd3a55d59d2ff5839ec (table test-table [id=d4afd141e1dd49b3a90944c2eedb799a]) successfully deleted
Aug 11 17:19:12 dist-test-slave-mjjr krb5kdc[11631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468751, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.157.129@KRBTEST.COM
I20260811 17:19:12.693109 12575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:12.683170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40746 (local address 127.9.157.129:37597)
0811 17:19:12.683296 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:12.683301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:12.683740 (+ 439us) server_negotiation.cc:407] Connection header received
0811 17:19:12.683953 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:12.683959 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:12.684024 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:12.684078 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:12.685082 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.685661 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.686559 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.686783 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.686919 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:12.689566 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:12.689578 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:12.689594 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:12.689619 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:12.691159 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:12.691703 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:12.691709 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:12.691710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:12.691796 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:12.692101 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:12.692105 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:12.692106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:12.692373 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:12.692604 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:12.692781 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 17:19:12.692934 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 17:19:12.693660 12046 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40746
I20260811 17:19:12.702502 11809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:12.695329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45084 (local address 127.9.157.190:41783)
0811 17:19:12.695459 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:12.695469 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:12.695484 (+ 15us) server_negotiation.cc:407] Connection header received
0811 17:19:12.695609 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:12.695611 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:12.695695 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:12.695773 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:12.696411 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.696949 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.697780 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:12.698010 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:12.698095 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:12.699461 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:12.699464 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:12.699466 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:12.699490 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:12.700994 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:12.701382 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:12.701384 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:12.701386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:12.701458 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:12.701682 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:12.701684 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:12.701685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:12.701854 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:12.701960 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:12.702269 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 17:19:12.702378 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 17:19:12.702955 11713 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45084
I20260811 17:19:12.704324 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 11843
I20260811 17:19:12.727012 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 12069
I20260811 17:19:12.748003 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 12322
I20260811 17:19:12.760936 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 11658
2026-08-11T17:19:12Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3336 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:19:12 dist-test-slave-mjjr krb5kdc[12696](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:19:12 dist-test-slave-mjjr krb5kdc[12696](info): set up 2 sockets
Aug 11 17:19:12 dist-test-slave-mjjr krb5kdc[12696](info): commencing operation
krb5kdc: starting...
W20260811 17:19:14.049432 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:19:14 dist-test-slave-mjjr krb5kdc[12696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:19:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:19:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:19:14.190902 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:35395
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:41071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:35395
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:14.311750 12716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:14.312136 12716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:14.312232 12716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:14.316411 12716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:19:14.316533 12716 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 17:19:14.316565 12716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:14.316614 12716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:14.316666 12716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:19:14.316705 12716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:19:14.322280 12716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:41071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:35395
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:35395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:14.323532 12716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:14.324687 12716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:14.333918 12725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:14.333972 12716 server_base.cc:1034] running on GCE node
W20260811 17:19:14.333940 12733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:14.333918 12726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:14.334830 12716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:14.336102 12716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:14.337287 12716 hybrid_clock.cc:648] HybridClock initialized: now 1786468754337267 us; error 37 us; skew 500 ppm
Aug 11 17:19:14 dist-test-slave-mjjr krb5kdc[12696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468754, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:14.345168 12716 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:19:14.346306 12716 webserver.cc:460] Webserver started at http://127.9.157.190:46607/ using document root <none> and password file <none>
I20260811 17:19:14.346951 12716 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:14.347021 12716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:14.347209 12716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:14.349527 12716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "b4346e5ce39a460589fbe00e6188619f"
format_stamp: "Formatted at 2026-08-11 17:19:14 on dist-test-slave-mjjr"
server_key: "634e969b47e7e346de397712db5ff59c"
server_key_iv: "93d8dd45d12772afbaa325edb1b08bdd"
server_key_version: "encryptionkey@0"
I20260811 17:19:14.350178 12716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "b4346e5ce39a460589fbe00e6188619f"
format_stamp: "Formatted at 2026-08-11 17:19:14 on dist-test-slave-mjjr"
server_key: "634e969b47e7e346de397712db5ff59c"
server_key_iv: "93d8dd45d12772afbaa325edb1b08bdd"
server_key_version: "encryptionkey@0"
I20260811 17:19:14.354449 12716 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260811 17:19:14.357123 12743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:14.358484 12716 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:19:14.358603 12716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "b4346e5ce39a460589fbe00e6188619f"
format_stamp: "Formatted at 2026-08-11 17:19:14 on dist-test-slave-mjjr"
server_key: "634e969b47e7e346de397712db5ff59c"
server_key_iv: "93d8dd45d12772afbaa325edb1b08bdd"
server_key_version: "encryptionkey@0"
I20260811 17:19:14.358739 12716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:14.378755 12716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:14.382109 12716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:14.382356 12716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:14.391185 12716 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:35395
I20260811 17:19:14.391196 12844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:35395 every 8 connection(s)
I20260811 17:19:14.392230 12716 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:19:14.395264 12845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:14.399619 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 12716
I20260811 17:19:14.399781 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:19:14.400060 9846 external_mini_cluster.cc:1428] Setting key 4964bcb16dcdc96cf4135d38f175dfb6
I20260811 17:19:14.401816 12845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4346e5ce39a460589fbe00e6188619f: Bootstrap starting.
I20260811 17:19:14.404567 12845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4346e5ce39a460589fbe00e6188619f: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:14.405416 12845 log.cc:824] T 00000000000000000000000000000000 P b4346e5ce39a460589fbe00e6188619f: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:14.407941 12845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4346e5ce39a460589fbe00e6188619f: No bootstrap required, opened a new log
Aug 11 17:19:14 dist-test-slave-mjjr krb5kdc[12696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:14.412448 12845 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4346e5ce39a460589fbe00e6188619f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4346e5ce39a460589fbe00e6188619f" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 35395 } }
I20260811 17:19:14.412738 12845 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4346e5ce39a460589fbe00e6188619f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:14.412825 12845 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4346e5ce39a460589fbe00e6188619f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4346e5ce39a460589fbe00e6188619f, State: Initialized, Role: FOLLOWER
I20260811 17:19:14.413338 12845 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4346e5ce39a460589fbe00e6188619f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4346e5ce39a460589fbe00e6188619f" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 35395 } }
I20260811 17:19:14.413518 12845 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4346e5ce39a460589fbe00e6188619f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:19:14.413594 12845 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4346e5ce39a460589fbe00e6188619f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:19:14.413715 12845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4346e5ce39a460589fbe00e6188619f [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:14.415179 12845 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4346e5ce39a460589fbe00e6188619f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4346e5ce39a460589fbe00e6188619f" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 35395 } }
I20260811 17:19:14.415707 12845 leader_election.cc:302] T 00000000000000000000000000000000 P b4346e5ce39a460589fbe00e6188619f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4346e5ce39a460589fbe00e6188619f; no voters:
I20260811 17:19:14.416100 12845 leader_election.cc:288] T 00000000000000000000000000000000 P b4346e5ce39a460589fbe00e6188619f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:19:14.416258 12853 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4346e5ce39a460589fbe00e6188619f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:14.416548 12853 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4346e5ce39a460589fbe00e6188619f [term 1 LEADER]: Becoming Leader. State: Replica: b4346e5ce39a460589fbe00e6188619f, State: Running, Role: LEADER
I20260811 17:19:14.416887 12849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:14.401650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48770 (local address 127.9.157.190:35395)
0811 17:19:14.402169 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:14.402193 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:14.402234 (+ 41us) server_negotiation.cc:407] Connection header received
0811 17:19:14.402932 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:14.402949 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:14.403306 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:14.403609 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:14.404648 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:14.405897 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:14.407098 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:14.407621 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:14.407844 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:14.410804 (+ 2960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:14.410810 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:14.410827 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:14.410865 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:14.412808 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:14.413402 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:14.413408 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:14.413414 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:14.413588 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:14.413956 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:14.413960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:14.413963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:14.414554 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:14.414796 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:14.415194 (+ 398us) server_negotiation.cc:299] Negotiation successful
0811 17:19:14.415584 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":169,"threads_started":1}
I20260811 17:19:14.416968 12853 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4346e5ce39a460589fbe00e6188619f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4346e5ce39a460589fbe00e6188619f" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 35395 } }
I20260811 17:19:14.417536 12845 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4346e5ce39a460589fbe00e6188619f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:19:14.418975 12854 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4346e5ce39a460589fbe00e6188619f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4346e5ce39a460589fbe00e6188619f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4346e5ce39a460589fbe00e6188619f" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 35395 } } }
I20260811 17:19:14.419127 12854 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4346e5ce39a460589fbe00e6188619f [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:14.419574 12855 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4346e5ce39a460589fbe00e6188619f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4346e5ce39a460589fbe00e6188619f. Latest consensus state: current_term: 1 leader_uuid: "b4346e5ce39a460589fbe00e6188619f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4346e5ce39a460589fbe00e6188619f" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 35395 } } }
I20260811 17:19:14.419788 12855 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4346e5ce39a460589fbe00e6188619f [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:14.420142 12869 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:19:14.423622 12869 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:19:14.429924 12869 catalog_manager.cc:1347] Generated new cluster ID: ef61cd47007541308219799bbfb71aaf
I20260811 17:19:14.430022 12869 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:19:14.446749 12869 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:19:14.447925 12869 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:19:14.458630 12869 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4346e5ce39a460589fbe00e6188619f: Generated new TSK 0
I20260811 17:19:14.459580 12869 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
I20260811 17:19:14.495437 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:0
--local_ip_for_outbound_sockets=127.9.157.129
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:35395
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:41071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:14.639304 12891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:14.639658 12891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:14.639784 12891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:14.643955 12891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:14.644050 12891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:14.644126 12891 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 17:19:14.644169 12891 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 17:19:14.644237 12891 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:19:14.649767 12891 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:41071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.157.190:35395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:14.650979 12891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:14.652170 12891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:14.662801 12905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:14.662834 12898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:14.662828 12903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:14.663025 12891 server_base.cc:1034] running on GCE node
I20260811 17:19:14.663767 12891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:14.664515 12891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:14.665729 12891 hybrid_clock.cc:648] HybridClock initialized: now 1786468754665687 us; error 51 us; skew 500 ppm
Aug 11 17:19:14 dist-test-slave-mjjr krb5kdc[12696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468754, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:14.668978 12891 init.cc:377] Logged in from keytab as kudu/127.9.157.129@KRBTEST.COM (short username kudu)
I20260811 17:19:14.670069 12891 webserver.cc:460] Webserver started at http://127.9.157.129:33583/ using document root <none> and password file <none>
I20260811 17:19:14.670743 12891 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:14.670851 12891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:14.671132 12891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:14.673555 12891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-0/data/instance:
uuid: "70e86f03eaec40a6b498346aad09c6a5"
format_stamp: "Formatted at 2026-08-11 17:19:14 on dist-test-slave-mjjr"
server_key: "8994683d347dcc9e8bae3c43392416f0"
server_key_iv: "a61be64c012e22a00c2a5313dc084de5"
server_key_version: "encryptionkey@0"
I20260811 17:19:14.674233 12891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance:
uuid: "70e86f03eaec40a6b498346aad09c6a5"
format_stamp: "Formatted at 2026-08-11 17:19:14 on dist-test-slave-mjjr"
server_key: "8994683d347dcc9e8bae3c43392416f0"
server_key_iv: "a61be64c012e22a00c2a5313dc084de5"
server_key_version: "encryptionkey@0"
I20260811 17:19:14.678798 12891 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 17:19:14.681761 12918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:14.683040 12891 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 17:19:14.683207 12891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "70e86f03eaec40a6b498346aad09c6a5"
format_stamp: "Formatted at 2026-08-11 17:19:14 on dist-test-slave-mjjr"
server_key: "8994683d347dcc9e8bae3c43392416f0"
server_key_iv: "a61be64c012e22a00c2a5313dc084de5"
server_key_version: "encryptionkey@0"
I20260811 17:19:14.683346 12891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:14.705878 12891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:14.709757 12891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:14.710001 12891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:14.711627 12891 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:14.711732 12891 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:14.711831 12891 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:14.711867 12891 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 17:19:14 dist-test-slave-mjjr krb5kdc[12696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468754, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:14.723842 12891 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:39375
I20260811 17:19:14.724922 12891 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:19:14.725855 13124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:39375 every 8 connection(s)
I20260811 17:19:14.726455 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 12891
I20260811 17:19:14.726595 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance
I20260811 17:19:14.726826 12849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:14.712703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:52053 (local address 127.9.157.190:35395)
0811 17:19:14.712929 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:14.712935 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:14.713778 (+ 843us) server_negotiation.cc:407] Connection header received
0811 17:19:14.714946 (+ 1168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:14.714951 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:14.715024 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:14.715146 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:14.716487 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:14.717133 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:14.718739 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:14.719099 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:14.719224 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:14.722575 (+ 3351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:14.722580 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:14.722582 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:14.722618 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:14.724333 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:14.724928 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:14.724932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:14.724934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:14.725060 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:14.725464 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:14.725468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:14.725470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:14.725689 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:14.725812 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:14.726491 (+ 679us) server_negotiation.cc:299] Negotiation successful
0811 17:19:14.726643 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260811 17:19:14.726919 9846 external_mini_cluster.cc:1428] Setting key a3be42171e57e6b4a1841669130e3cda
I20260811 17:19:14.727907 12936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:14.713104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:35395 (local address 127.9.157.129:52053)
0811 17:19:14.713586 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:14.713641 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:14.714626 (+ 985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:14.715370 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:14.715377 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:14.715813 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:14.716317 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:14.716330 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:14.717279 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:14.717299 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:14.718602 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:14.718613 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:14.719300 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:14.719305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:14.719539 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:14.720320 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:14.720354 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:14.722354 (+ 2000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:14.724481 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:14.724488 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:14.724506 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:14.724796 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:14.725164 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:14.725168 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:14.725170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:14.725352 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:14.725870 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:14.725877 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:14.726191 (+ 314us) client_negotiation.cc:769] Sending connection context
0811 17:19:14.726429 (+ 238us) client_negotiation.cc:240] Negotiation successful
0811 17:19:14.726796 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":138,"threads_started":1}
I20260811 17:19:14.729470 13126 heartbeater.cc:347] Connected to a master server at 127.9.157.190:35395
I20260811 17:19:14.729626 13126 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:14.730247 13126 heartbeater.cc:511] Master 127.9.157.190:35395 requested a full tablet report, sending...
I20260811 17:19:14.732208 12772 ts_manager.cc:194] Registered new tserver with Master: 70e86f03eaec40a6b498346aad09c6a5 (127.9.157.129:39375)
I20260811 17:19:14.733932 12772 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.129@KRBTEST.COM'} at 127.9.157.129:52053
WARNING: no policy specified for kudu/127.9.157.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.130@KRBTEST.COM" created.
I20260811 17:19:14.743830 12849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:14.731994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:44299 (local address 127.9.157.190:35395)
0811 17:19:14.732152 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:14.732157 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:14.735046 (+ 2889us) server_negotiation.cc:407] Connection header received
0811 17:19:14.735270 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:14.735276 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:14.735339 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:14.735399 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:14.736267 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:14.737028 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:14.738380 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:14.738609 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:14.738713 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:14.740247 (+ 1534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:14.740251 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:14.740253 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:14.740297 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:14.741825 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:14.742351 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:14.742353 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:14.742355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:14.742440 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:14.742745 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:14.742749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:14.742750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:14.743039 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:14.743177 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:14.743488 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 17:19:14.743611 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 17:19:14.743844 13136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:14.731947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:35395 (local address 127.9.157.129:44299)
0811 17:19:14.734933 (+ 2986us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:14.734955 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:14.735145 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:14.735572 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:14.735574 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:14.735897 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:14.736091 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:14.736104 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:14.737215 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:14.737220 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:14.738226 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:14.738240 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:14.738771 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:14.738779 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:14.738907 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:14.739530 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:14.739551 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:14.740065 (+ 514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:14.741951 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:14.741954 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:14.741955 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:14.742220 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:14.742530 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:14.742533 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:14.742534 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:14.742617 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:14.743163 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:14.743167 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:14.743298 (+ 131us) client_negotiation.cc:769] Sending connection context
0811 17:19:14.743422 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 17:19:14.743611 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2845,"mutex_wait_us":1,"thread_start_us":144,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
I20260811 17:19:14.760267 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:0
--local_ip_for_outbound_sockets=127.9.157.130
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:35395
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:41071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:14.885567 13138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:14.885875 13138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:14.885931 13138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:14.890360 13138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:14.890434 13138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:14.890476 13138 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 17:19:14.890517 13138 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 17:19:14.890579 13138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:19:14.896376 13138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:41071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.157.190:35395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:14.897619 13138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:14.898801 13138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:14.909437 13150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:14.909539 13138 server_base.cc:1034] running on GCE node
W20260811 17:19:14.909482 13149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:14.909449 13153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:14.910207 13138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:14.910990 13138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:14.912235 13138 hybrid_clock.cc:648] HybridClock initialized: now 1786468754912215 us; error 33 us; skew 500 ppm
Aug 11 17:19:14 dist-test-slave-mjjr krb5kdc[12696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468754, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:14.915340 13138 init.cc:377] Logged in from keytab as kudu/127.9.157.130@KRBTEST.COM (short username kudu)
I20260811 17:19:14.916472 13138 webserver.cc:460] Webserver started at http://127.9.157.130:44669/ using document root <none> and password file <none>
I20260811 17:19:14.917109 13138 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:14.917172 13138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:14.917409 13138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:14.920008 13138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-1/data/instance:
uuid: "e87af54a41a547d0bc28454d0cee2a06"
format_stamp: "Formatted at 2026-08-11 17:19:14 on dist-test-slave-mjjr"
server_key: "296a0ae47617b5e0cb870b787026c077"
server_key_iv: "7d9ae4a75be9b5f6a8e351be20bc650b"
server_key_version: "encryptionkey@0"
I20260811 17:19:14.920722 13138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance:
uuid: "e87af54a41a547d0bc28454d0cee2a06"
format_stamp: "Formatted at 2026-08-11 17:19:14 on dist-test-slave-mjjr"
server_key: "296a0ae47617b5e0cb870b787026c077"
server_key_iv: "7d9ae4a75be9b5f6a8e351be20bc650b"
server_key_version: "encryptionkey@0"
I20260811 17:19:14.924891 13138 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 17:19:14.927731 13163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:14.929253 13138 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 17:19:14.929414 13138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "e87af54a41a547d0bc28454d0cee2a06"
format_stamp: "Formatted at 2026-08-11 17:19:14 on dist-test-slave-mjjr"
server_key: "296a0ae47617b5e0cb870b787026c077"
server_key_iv: "7d9ae4a75be9b5f6a8e351be20bc650b"
server_key_version: "encryptionkey@0"
I20260811 17:19:14.929569 13138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:14.952998 13138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:14.956704 13138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:14.956945 13138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:14.958724 13138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:14.958806 13138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:14.958878 13138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:14.958933 13138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 17:19:14 dist-test-slave-mjjr krb5kdc[12696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468754, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:14.970685 13138 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:35635
I20260811 17:19:14.970736 13353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:35635 every 8 connection(s)
I20260811 17:19:14.972049 13138 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:19:14.974654 12849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:14.959783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:34375 (local address 127.9.157.190:35395)
0811 17:19:14.959924 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:14.959928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:14.961110 (+ 1182us) server_negotiation.cc:407] Connection header received
0811 17:19:14.962303 (+ 1193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:14.962306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:14.962366 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:14.962457 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:14.964334 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:14.965175 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:14.966448 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:14.966691 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:14.966812 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:14.970414 (+ 3602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:14.970419 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:14.970423 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:14.970463 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:14.972334 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:14.973032 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:14.973035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:14.973036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:14.973120 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:14.973487 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:14.973490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:14.973491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:14.973692 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:14.973804 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:14.974401 (+ 597us) server_negotiation.cc:299] Negotiation successful
0811 17:19:14.974526 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 17:19:14.975483 13180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:14.960408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:35395 (local address 127.9.157.130:34375)
0811 17:19:14.960935 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:14.960983 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:14.962041 (+ 1058us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:14.962691 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:14.962701 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:14.963299 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:14.964132 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:14.964159 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:14.965320 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:14.965324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:14.966307 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:14.966317 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:14.966873 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:14.966877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:14.967049 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:14.968376 (+ 1327us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:14.968394 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:14.970161 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:14.972482 (+ 2321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:14.972489 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:14.972509 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:14.972888 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:14.973199 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:14.973203 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:14.973204 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:14.973373 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:14.973856 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:14.973862 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:14.974125 (+ 263us) client_negotiation.cc:769] Sending connection context
0811 17:19:14.974294 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 17:19:14.974567 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":142,"threads_started":1}
I20260811 17:19:14.976814 13354 heartbeater.cc:347] Connected to a master server at 127.9.157.190:35395
I20260811 17:19:14.976998 13354 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:14.977540 13354 heartbeater.cc:511] Master 127.9.157.190:35395 requested a full tablet report, sending...
I20260811 17:19:14.978782 12773 ts_manager.cc:194] Registered new tserver with Master: e87af54a41a547d0bc28454d0cee2a06 (127.9.157.130:35635)
I20260811 17:19:14.979610 12773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.130@KRBTEST.COM'} at 127.9.157.130:34375
I20260811 17:19:14.979717 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 13138
I20260811 17:19:14.979833 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance
I20260811 17:19:14.980194 9846 external_mini_cluster.cc:1428] Setting key 034020ce5c3d9fcae1ad21525a0cea5d
I20260811 17:19:14.987521 12849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:14.979283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:52711 (local address 127.9.157.190:35395)
0811 17:19:14.979791 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:14.979796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:14.979815 (+ 19us) server_negotiation.cc:407] Connection header received
0811 17:19:14.979936 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:14.979939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:14.979990 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:14.980080 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:14.980798 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:14.981405 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:14.982270 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:14.982658 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:14.982787 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:14.984116 (+ 1329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:14.984119 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:14.984121 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:14.984144 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:14.985540 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:14.986068 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:14.986070 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:14.986072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:14.986149 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:14.986483 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:14.986485 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:14.986486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:14.986696 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:14.986774 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:14.987225 (+ 451us) server_negotiation.cc:299] Negotiation successful
0811 17:19:14.987346 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416}
I20260811 17:19:14.987612 13364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:14.979168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:35395 (local address 127.9.157.130:52711)
0811 17:19:14.979719 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:14.979735 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:14.979831 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:14.980101 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:14.980104 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:14.980345 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:14.980483 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:14.980490 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:14.981573 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:14.981576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:14.982147 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:14.982155 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:14.982794 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:14.982797 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:14.982879 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:14.983440 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:14.983451 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:14.983939 (+ 488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:14.985660 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:14.985666 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:14.985671 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:14.985955 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:14.986261 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:14.986267 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:14.986271 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:14.986381 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:14.986798 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:14.986805 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:14.986949 (+ 144us) client_negotiation.cc:769] Sending connection context
0811 17:19:14.987093 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 17:19:14.987348 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":10,"thread_start_us":106,"threads_started":1}
WARNING: no policy specified for kudu/127.9.157.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
I20260811 17:19:15.012315 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:0
--local_ip_for_outbound_sockets=127.9.157.131
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:35395
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:41071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:15.136485 13371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:15.136782 13371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:15.136832 13371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:15.140930 13371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:15.140993 13371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:15.141036 13371 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 17:19:15.141049 13371 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 17:19:15.141083 13371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:19:15.146603 13371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:41071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.157.190:35395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:15.147874 13371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:15.148947 13371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:15.159260 13387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:15.159291 13382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:15.159348 13380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:15.159614 13371 server_base.cc:1034] running on GCE node
I20260811 17:19:15.160174 13371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:15.160899 13371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:15.162104 13371 hybrid_clock.cc:648] HybridClock initialized: now 1786468755162093 us; error 35 us; skew 500 ppm
Aug 11 17:19:15 dist-test-slave-mjjr krb5kdc[12696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468755, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:15.165087 13371 init.cc:377] Logged in from keytab as kudu/127.9.157.131@KRBTEST.COM (short username kudu)
I20260811 17:19:15.166067 13371 webserver.cc:460] Webserver started at http://127.9.157.131:33521/ using document root <none> and password file <none>
I20260811 17:19:15.166711 13371 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:15.166811 13371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:15.167066 13371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:15.169348 13371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-2/data/instance:
uuid: "61a52b81dbce461eb4a07db7c63fc6e5"
format_stamp: "Formatted at 2026-08-11 17:19:15 on dist-test-slave-mjjr"
server_key: "4085b1ff72354d9d06b84b2dda5930c9"
server_key_iv: "358a541750e613cf71d082c40ec9d078"
server_key_version: "encryptionkey@0"
I20260811 17:19:15.169991 13371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance:
uuid: "61a52b81dbce461eb4a07db7c63fc6e5"
format_stamp: "Formatted at 2026-08-11 17:19:15 on dist-test-slave-mjjr"
server_key: "4085b1ff72354d9d06b84b2dda5930c9"
server_key_iv: "358a541750e613cf71d082c40ec9d078"
server_key_version: "encryptionkey@0"
I20260811 17:19:15.174368 13371 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 17:19:15.177203 13396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:15.178913 13371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:19:15.179085 13371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "61a52b81dbce461eb4a07db7c63fc6e5"
format_stamp: "Formatted at 2026-08-11 17:19:15 on dist-test-slave-mjjr"
server_key: "4085b1ff72354d9d06b84b2dda5930c9"
server_key_iv: "358a541750e613cf71d082c40ec9d078"
server_key_version: "encryptionkey@0"
I20260811 17:19:15.179221 13371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:15.189935 13371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:15.193416 13371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:15.193651 13371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:15.195439 13371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:15.195534 13371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:15.195632 13371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:15.195730 13371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 11 17:19:15 dist-test-slave-mjjr krb5kdc[12696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468755, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:15.208432 13371 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:36667
I20260811 17:19:15.208766 13587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:36667 every 8 connection(s)
I20260811 17:19:15.209717 13371 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:19:15.210716 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 13371
I20260811 17:19:15.210863 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance
I20260811 17:19:15.211170 9846 external_mini_cluster.cc:1428] Setting key 6aaf9bd5581f67b72c926107f0731ae3
I20260811 17:19:15.211880 12849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.196219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:49497 (local address 127.9.157.190:35395)
0811 17:19:15.196389 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:15.196394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:15.197502 (+ 1108us) server_negotiation.cc:407] Connection header received
0811 17:19:15.198874 (+ 1372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:15.198878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:15.198942 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:15.199036 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:15.200654 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.201199 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.202522 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.202953 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.203095 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.207382 (+ 4287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:15.207386 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:15.207388 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:15.207421 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:15.209270 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:15.209974 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:15.209977 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:15.209979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:15.210070 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:15.210563 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:15.210569 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:15.210572 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:15.210844 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:15.210947 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:15.211530 (+ 583us) server_negotiation.cc:299] Negotiation successful
0811 17:19:15.211711 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 17:19:15.212993 13412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.196723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:35395 (local address 127.9.157.131:49497)
0811 17:19:15.197285 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:15.197351 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:15.198569 (+ 1218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:15.199331 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:15.199344 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:15.199802 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:15.200471 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:15.200490 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.201352 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.201355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:15.202353 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:15.202367 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.203142 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.203149 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:15.203400 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.204189 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:15.204219 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:15.207116 (+ 2897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:15.209413 (+ 2297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:15.209428 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:15.209445 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:15.209826 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:15.210203 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:15.210207 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:15.210212 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:15.210429 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:15.210951 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:15.210960 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:15.211229 (+ 269us) client_negotiation.cc:769] Sending connection context
0811 17:19:15.211484 (+ 255us) client_negotiation.cc:240] Negotiation successful
0811 17:19:15.211883 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":122,"threads_started":1}
I20260811 17:19:15.214414 13592 heartbeater.cc:347] Connected to a master server at 127.9.157.190:35395
I20260811 17:19:15.214599 13592 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:15.215196 13592 heartbeater.cc:511] Master 127.9.157.190:35395 requested a full tablet report, sending...
I20260811 17:19:15.216398 12772 ts_manager.cc:194] Registered new tserver with Master: 61a52b81dbce461eb4a07db7c63fc6e5 (127.9.157.131:36667)
I20260811 17:19:15.217228 12772 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.131@KRBTEST.COM'} at 127.9.157.131:49497
I20260811 17:19:15.224547 12849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.216789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:55291 (local address 127.9.157.190:35395)
0811 17:19:15.216941 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:15.216945 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:15.217096 (+ 151us) server_negotiation.cc:407] Connection header received
0811 17:19:15.217198 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:15.217201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:15.217252 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:15.217357 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:15.217924 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.218621 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.219562 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.219875 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.219993 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.221186 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:15.221190 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:15.221193 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:15.221227 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:15.222720 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:15.223180 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:15.223184 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:15.223186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:15.223275 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:15.223570 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:15.223573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:15.223575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:15.223856 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:15.223971 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:15.224251 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 17:19:15.224370 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 17:19:15.224563 13604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.216734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:35395 (local address 127.9.157.131:55291)
0811 17:19:15.217016 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:15.217029 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:15.217115 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:15.217479 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:15.217484 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:15.217699 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:15.217804 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:15.217810 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.218753 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.218763 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:15.219448 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:15.219458 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.219999 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.220003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:15.220110 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.220672 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:15.220685 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:15.221029 (+ 344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:15.222847 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:15.222851 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:15.222854 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:15.223074 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:15.223376 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:15.223380 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:15.223382 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:15.223472 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:15.223962 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:15.223966 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:15.224077 (+ 111us) client_negotiation.cc:769] Sending connection context
0811 17:19:15.224190 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 17:19:15.224364 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":108,"threads_started":1}
I20260811 17:19:15.225634 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 17:19:15 dist-test-slave-mjjr krb5kdc[12696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 17:19:15 dist-test-slave-mjjr krb5kdc[12696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:15.255935 12849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.247102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48778 (local address 127.9.157.190:35395)
0811 17:19:15.247246 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:15.247250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:15.247273 (+ 23us) server_negotiation.cc:407] Connection header received
0811 17:19:15.247318 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:15.247321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:15.247364 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:15.247486 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:15.248363 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.248900 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.249726 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.249950 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.250031 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.252667 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:15.252671 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:15.252673 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:15.252714 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:15.254174 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:15.254636 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:15.254639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:15.254640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:15.254719 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:15.254964 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:15.254966 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:15.254967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:15.255158 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:15.255261 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:15.255552 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 17:19:15.255749 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 17:19:15.259617 12773 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48778:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:19:15.263157 12773 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:19:15.279983 13623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.274224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50316 (local address 127.9.157.129:39375)
0811 17:19:15.274546 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:15.274554 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:15.274578 (+ 24us) server_negotiation.cc:407] Connection header received
0811 17:19:15.274657 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:15.274660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:15.274858 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:15.275003 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:15.275449 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.276616 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.278597 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.279170 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.279313 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.279475 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:15.279713 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 17:19:15.279825 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":109,"threads_started":1}
I20260811 17:19:15.279976 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.272945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:39375 (local address 127.0.0.1:50316)
0811 17:19:15.273761 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:15.273819 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:15.273968 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:15.275161 (+ 1193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:15.275164 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:15.275183 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:15.275299 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:15.275306 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.276727 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.276731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:15.278471 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:15.278481 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.279318 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.279322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:15.279449 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.279495 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 17:19:15.279636 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 17:19:15.279813 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"mutex_wait_us":492,"spinlock_wait_cycles":6656,"thread_start_us":121,"threads_started":1}
I20260811 17:19:15.280169 13622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.273687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:36667 (local address 127.0.0.1:55686)
0811 17:19:15.273937 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:15.273952 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:15.274059 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:15.275406 (+ 1347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:15.275409 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:15.275428 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:15.275539 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:15.275549 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.276606 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.276610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:15.278517 (+ 1907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:15.278529 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.279784 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.279790 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:15.279889 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.279911 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 17:19:15.279969 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 17:19:15.280038 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":566,"thread_start_us":115,"threads_started":1}
I20260811 17:19:15.280155 13620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.273030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54808 (local address 127.9.157.130:35635)
0811 17:19:15.273435 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:15.273448 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:15.274118 (+ 670us) server_negotiation.cc:407] Connection header received
0811 17:19:15.274210 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:15.274226 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:15.274458 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:15.274621 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:15.275161 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.276427 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.278292 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.279170 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.279306 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.279478 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:15.279859 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 17:19:15.279977 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":123,"threads_started":1}
I20260811 17:19:15.280725 13625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.274273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55686 (local address 127.9.157.131:36667)
0811 17:19:15.274976 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:15.274989 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:15.275022 (+ 33us) server_negotiation.cc:407] Connection header received
0811 17:19:15.275102 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:15.275109 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:15.275264 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:15.275430 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:15.275689 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.276502 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.278682 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.279483 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.279591 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.280422 (+ 831us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:15.280476 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 17:19:15.280564 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":134,"threads_started":1}
I20260811 17:19:15.281021 13619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.272888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:35635 (local address 127.0.0.1:54808)
0811 17:19:15.273761 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:15.273819 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:15.273968 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:15.274791 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:15.274797 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:15.274821 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:15.274976 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:15.274985 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.276552 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.276557 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:15.278147 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:15.278158 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.279554 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.279560 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:15.279691 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.279714 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 17:19:15.280325 (+ 611us) client_negotiation.cc:240] Negotiation successful
0811 17:19:15.280401 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"mutex_wait_us":56,"spinlock_wait_cycles":4096,"thread_start_us":103,"threads_started":1}
I20260811 17:19:15.283927 13474 tablet_service.cc:1467] Processing CreateTablet for tablet 73e92b26cad94cdabe3d25396913be76 (DEFAULT_TABLE table=test-table [id=f935ed3bc56943fd89632a9521aeeff4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:15.283926 13000 tablet_service.cc:1467] Processing CreateTablet for tablet 73e92b26cad94cdabe3d25396913be76 (DEFAULT_TABLE table=test-table [id=f935ed3bc56943fd89632a9521aeeff4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:15.283931 13254 tablet_service.cc:1467] Processing CreateTablet for tablet 73e92b26cad94cdabe3d25396913be76 (DEFAULT_TABLE table=test-table [id=f935ed3bc56943fd89632a9521aeeff4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:15.285099 13474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73e92b26cad94cdabe3d25396913be76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:15.285104 13000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73e92b26cad94cdabe3d25396913be76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:15.285104 13254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73e92b26cad94cdabe3d25396913be76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:15.292999 13627 tablet_bootstrap.cc:492] T 73e92b26cad94cdabe3d25396913be76 P 70e86f03eaec40a6b498346aad09c6a5: Bootstrap starting.
I20260811 17:19:15.293953 13630 tablet_bootstrap.cc:492] T 73e92b26cad94cdabe3d25396913be76 P e87af54a41a547d0bc28454d0cee2a06: Bootstrap starting.
I20260811 17:19:15.294404 13629 tablet_bootstrap.cc:492] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5: Bootstrap starting.
I20260811 17:19:15.295586 13627 tablet_bootstrap.cc:654] T 73e92b26cad94cdabe3d25396913be76 P 70e86f03eaec40a6b498346aad09c6a5: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:15.296581 13629 tablet_bootstrap.cc:654] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:15.296593 13627 log.cc:824] T 73e92b26cad94cdabe3d25396913be76 P 70e86f03eaec40a6b498346aad09c6a5: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:15.297596 13629 log.cc:824] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:15.298065 13630 tablet_bootstrap.cc:654] T 73e92b26cad94cdabe3d25396913be76 P e87af54a41a547d0bc28454d0cee2a06: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:15.298903 13630 log.cc:824] T 73e92b26cad94cdabe3d25396913be76 P e87af54a41a547d0bc28454d0cee2a06: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:15.298960 13627 tablet_bootstrap.cc:492] T 73e92b26cad94cdabe3d25396913be76 P 70e86f03eaec40a6b498346aad09c6a5: No bootstrap required, opened a new log
I20260811 17:19:15.299206 13627 ts_tablet_manager.cc:1397] T 73e92b26cad94cdabe3d25396913be76 P 70e86f03eaec40a6b498346aad09c6a5: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 17:19:15.299882 13629 tablet_bootstrap.cc:492] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5: No bootstrap required, opened a new log
I20260811 17:19:15.300101 13629 ts_tablet_manager.cc:1397] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 17:19:15.301216 13630 tablet_bootstrap.cc:492] T 73e92b26cad94cdabe3d25396913be76 P e87af54a41a547d0bc28454d0cee2a06: No bootstrap required, opened a new log
I20260811 17:19:15.301429 13630 ts_tablet_manager.cc:1397] T 73e92b26cad94cdabe3d25396913be76 P e87af54a41a547d0bc28454d0cee2a06: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260811 17:19:15.303443 13627 raft_consensus.cc:348] T 73e92b26cad94cdabe3d25396913be76 P 70e86f03eaec40a6b498346aad09c6a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:15.303820 13627 raft_consensus.cc:374] T 73e92b26cad94cdabe3d25396913be76 P 70e86f03eaec40a6b498346aad09c6a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:15.303912 13627 raft_consensus.cc:729] T 73e92b26cad94cdabe3d25396913be76 P 70e86f03eaec40a6b498346aad09c6a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70e86f03eaec40a6b498346aad09c6a5, State: Initialized, Role: FOLLOWER
I20260811 17:19:15.304226 13629 raft_consensus.cc:348] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:15.304487 13629 raft_consensus.cc:374] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:15.304406 13627 consensus_queue.cc:260] T 73e92b26cad94cdabe3d25396913be76 P 70e86f03eaec40a6b498346aad09c6a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:15.304580 13629 raft_consensus.cc:729] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61a52b81dbce461eb4a07db7c63fc6e5, State: Initialized, Role: FOLLOWER
I20260811 17:19:15.305059 13629 consensus_queue.cc:260] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:15.305238 13126 heartbeater.cc:503] Master 127.9.157.190:35395 was elected leader, sending a full tablet report...
I20260811 17:19:15.305645 13627 ts_tablet_manager.cc:1428] T 73e92b26cad94cdabe3d25396913be76 P 70e86f03eaec40a6b498346aad09c6a5: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 17:19:15.305816 13592 heartbeater.cc:503] Master 127.9.157.190:35395 was elected leader, sending a full tablet report...
I20260811 17:19:15.307055 13630 raft_consensus.cc:348] T 73e92b26cad94cdabe3d25396913be76 P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:15.307376 13630 raft_consensus.cc:374] T 73e92b26cad94cdabe3d25396913be76 P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:15.307454 13630 raft_consensus.cc:729] T 73e92b26cad94cdabe3d25396913be76 P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e87af54a41a547d0bc28454d0cee2a06, State: Initialized, Role: FOLLOWER
I20260811 17:19:15.307583 13629 ts_tablet_manager.cc:1428] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 17:19:15.307950 13630 consensus_queue.cc:260] T 73e92b26cad94cdabe3d25396913be76 P e87af54a41a547d0bc28454d0cee2a06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:15.308674 13354 heartbeater.cc:503] Master 127.9.157.190:35395 was elected leader, sending a full tablet report...
I20260811 17:19:15.309053 13630 ts_tablet_manager.cc:1428] T 73e92b26cad94cdabe3d25396913be76 P e87af54a41a547d0bc28454d0cee2a06: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260811 17:19:15.341254 13638 raft_consensus.cc:482] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:19:15.341548 13638 raft_consensus.cc:504] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:15.343051 13638 leader_election.cc:288] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e87af54a41a547d0bc28454d0cee2a06 (127.9.157.130:35635), 70e86f03eaec40a6b498346aad09c6a5 (127.9.157.129:39375)
I20260811 17:19:15.347303 13412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.343333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:39375 (local address 127.9.157.131:39931)
0811 17:19:15.343504 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:15.343522 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:15.343622 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:15.344158 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:15.344161 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:15.344180 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:15.344310 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:15.344322 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.345164 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.345169 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:15.346112 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:15.346123 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.346837 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.346841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:15.346956 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.346976 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 17:19:15.347068 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 17:19:15.347151 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 17:19:15.347303 13623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.343566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:39931 (local address 127.9.157.129:39375)
0811 17:19:15.343909 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:15.343915 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:15.343934 (+ 19us) server_negotiation.cc:407] Connection header received
0811 17:19:15.343996 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:15.343998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:15.344039 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:15.344142 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:15.344446 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.345051 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.346247 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.346702 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.346839 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.346896 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:15.347085 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 17:19:15.347165 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226}
I20260811 17:19:15.348030 13620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.343858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:55287 (local address 127.9.157.130:35635)
0811 17:19:15.344025 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:15.344030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:15.344046 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:19:15.344105 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:15.344108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:15.344154 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:15.344248 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:15.344532 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.345152 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.346869 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.347309 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.347456 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.347519 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:15.347838 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 17:19:15.347928 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 17:19:15.348135 13036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73e92b26cad94cdabe3d25396913be76" candidate_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70e86f03eaec40a6b498346aad09c6a5" is_pre_election: true
I20260811 17:19:15.348448 13641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.343361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:35635 (local address 127.9.157.131:55287)
0811 17:19:15.343874 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:15.343887 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:15.344021 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:15.344260 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:15.344263 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:15.344281 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:15.344413 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:15.344419 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.345270 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.345275 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:15.346727 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:15.346738 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.347550 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.347557 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:15.347680 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.347713 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 17:19:15.348205 (+ 492us) client_negotiation.cc:240] Negotiation successful
0811 17:19:15.348307 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":140,"threads_started":1}
I20260811 17:19:15.348546 13036 raft_consensus.cc:2454] T 73e92b26cad94cdabe3d25396913be76 P 70e86f03eaec40a6b498346aad09c6a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61a52b81dbce461eb4a07db7c63fc6e5 in term 0.
I20260811 17:19:15.349079 13266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73e92b26cad94cdabe3d25396913be76" candidate_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e87af54a41a547d0bc28454d0cee2a06" is_pre_election: true
I20260811 17:19:15.349105 13403 leader_election.cc:302] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61a52b81dbce461eb4a07db7c63fc6e5, 70e86f03eaec40a6b498346aad09c6a5; no voters:
I20260811 17:19:15.349385 13266 raft_consensus.cc:2454] T 73e92b26cad94cdabe3d25396913be76 P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61a52b81dbce461eb4a07db7c63fc6e5 in term 0.
I20260811 17:19:15.349432 13638 raft_consensus.cc:2781] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:19:15.349524 13638 raft_consensus.cc:482] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:19:15.349582 13638 raft_consensus.cc:3037] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:15.350878 13638 raft_consensus.cc:504] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:15.351282 13638 leader_election.cc:288] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5 [CANDIDATE]: Term 1 election: Requested vote from peers e87af54a41a547d0bc28454d0cee2a06 (127.9.157.130:35635), 70e86f03eaec40a6b498346aad09c6a5 (127.9.157.129:39375)
I20260811 17:19:15.351689 13266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73e92b26cad94cdabe3d25396913be76" candidate_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e87af54a41a547d0bc28454d0cee2a06"
I20260811 17:19:15.351826 13036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73e92b26cad94cdabe3d25396913be76" candidate_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70e86f03eaec40a6b498346aad09c6a5"
I20260811 17:19:15.351982 13036 raft_consensus.cc:3037] T 73e92b26cad94cdabe3d25396913be76 P 70e86f03eaec40a6b498346aad09c6a5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:15.351974 13266 raft_consensus.cc:3037] T 73e92b26cad94cdabe3d25396913be76 P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:15.353174 13266 raft_consensus.cc:2454] T 73e92b26cad94cdabe3d25396913be76 P e87af54a41a547d0bc28454d0cee2a06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61a52b81dbce461eb4a07db7c63fc6e5 in term 1.
I20260811 17:19:15.353175 13036 raft_consensus.cc:2454] T 73e92b26cad94cdabe3d25396913be76 P 70e86f03eaec40a6b498346aad09c6a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61a52b81dbce461eb4a07db7c63fc6e5 in term 1.
I20260811 17:19:15.353611 13401 leader_election.cc:302] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61a52b81dbce461eb4a07db7c63fc6e5, e87af54a41a547d0bc28454d0cee2a06; no voters:
I20260811 17:19:15.353826 13638 raft_consensus.cc:2781] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:15.354089 13638 raft_consensus.cc:686] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 1 LEADER]: Becoming Leader. State: Replica: 61a52b81dbce461eb4a07db7c63fc6e5, State: Running, Role: LEADER
I20260811 17:19:15.354503 13638 consensus_queue.cc:237] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:15.358147 12772 catalog_manager.cc:5495] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 61a52b81dbce461eb4a07db7c63fc6e5 (127.9.157.131). New cstate: current_term: 1 leader_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } health_report { overall_health: HEALTHY } } }
Aug 11 17:19:15 dist-test-slave-mjjr krb5kdc[12696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468754, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:15.419169 13657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.409651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:35395 (local address 127.0.0.1:48792)
0811 17:19:15.410080 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:15.410093 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:15.410249 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:15.410522 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:15.410525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:15.410982 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:15.411080 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:15.411086 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.411934 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.411941 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:15.412497 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:15.412505 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.412958 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.412961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:15.413058 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.413692 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:15.413743 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:15.415306 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:15.417239 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:15.417259 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:15.417266 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:15.417550 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:15.417950 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:15.417956 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:15.417959 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:15.418067 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:15.418516 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:15.418523 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:15.418658 (+ 135us) client_negotiation.cc:769] Sending connection context
0811 17:19:15.418805 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 17:19:15.418996 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":163,"threads_started":1}
I20260811 17:19:15.419214 12849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.409783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48792 (local address 127.9.157.190:35395)
0811 17:19:15.409973 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:15.409978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:15.410182 (+ 204us) server_negotiation.cc:407] Connection header received
0811 17:19:15.410376 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:15.410378 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:15.410416 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:15.410501 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:15.411211 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.411823 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.412621 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.412839 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.412950 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.415493 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:15.415496 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:15.415498 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:15.415525 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:15.417104 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:15.417671 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:15.417674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:15.417676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:15.417771 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:15.418160 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:15.418162 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:15.418163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:15.418370 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:15.418450 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:15.418977 (+ 527us) server_negotiation.cc:299] Negotiation successful
0811 17:19:15.419097 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":61}
I20260811 17:19:15.424077 12772 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:48792:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 17:19:15.424937 12772 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:19:15.430766 13474 tablet_service.cc:1467] Processing CreateTablet for tablet 696778f02edc4bd4a16747ce65f692cc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b9bc80b269ee4946bbf9fd1ee47cd80b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 17:19:15.430791 13254 tablet_service.cc:1467] Processing CreateTablet for tablet 696778f02edc4bd4a16747ce65f692cc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b9bc80b269ee4946bbf9fd1ee47cd80b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 17:19:15.431706 13474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 696778f02edc4bd4a16747ce65f692cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:15.431725 13254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 696778f02edc4bd4a16747ce65f692cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:15.434947 13629 tablet_bootstrap.cc:492] T 696778f02edc4bd4a16747ce65f692cc P 61a52b81dbce461eb4a07db7c63fc6e5: Bootstrap starting.
I20260811 17:19:15.434968 13630 tablet_bootstrap.cc:492] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06: Bootstrap starting.
I20260811 17:19:15.436084 13629 tablet_bootstrap.cc:654] T 696778f02edc4bd4a16747ce65f692cc P 61a52b81dbce461eb4a07db7c63fc6e5: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:15.435555 13000 tablet_service.cc:1467] Processing CreateTablet for tablet 696778f02edc4bd4a16747ce65f692cc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b9bc80b269ee4946bbf9fd1ee47cd80b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 17:19:15.436514 13630 tablet_bootstrap.cc:654] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:15.436524 13000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 696778f02edc4bd4a16747ce65f692cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:15.438030 13630 tablet_bootstrap.cc:492] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06: No bootstrap required, opened a new log
I20260811 17:19:15.438160 13630 ts_tablet_manager.cc:1397] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 17:19:15.438627 13630 raft_consensus.cc:348] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:15.438776 13630 raft_consensus.cc:374] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:15.438838 13630 raft_consensus.cc:729] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e87af54a41a547d0bc28454d0cee2a06, State: Initialized, Role: FOLLOWER
I20260811 17:19:15.439024 13630 consensus_queue.cc:260] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:15.439335 13630 ts_tablet_manager.cc:1428] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 17:19:15.439345 13639 tablet_replica.cc:439] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } } }
I20260811 17:19:15.439527 13639 tablet_replica.cc:442] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:19:15.440225 13627 tablet_bootstrap.cc:492] T 696778f02edc4bd4a16747ce65f692cc P 70e86f03eaec40a6b498346aad09c6a5: Bootstrap starting.
I20260811 17:19:15.441296 13629 tablet_bootstrap.cc:492] T 696778f02edc4bd4a16747ce65f692cc P 61a52b81dbce461eb4a07db7c63fc6e5: No bootstrap required, opened a new log
I20260811 17:19:15.441416 13627 tablet_bootstrap.cc:654] T 696778f02edc4bd4a16747ce65f692cc P 70e86f03eaec40a6b498346aad09c6a5: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:15.441416 13629 ts_tablet_manager.cc:1397] T 696778f02edc4bd4a16747ce65f692cc P 61a52b81dbce461eb4a07db7c63fc6e5: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260811 17:19:15.441926 13629 raft_consensus.cc:348] T 696778f02edc4bd4a16747ce65f692cc P 61a52b81dbce461eb4a07db7c63fc6e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:15.442072 13629 raft_consensus.cc:374] T 696778f02edc4bd4a16747ce65f692cc P 61a52b81dbce461eb4a07db7c63fc6e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:15.442121 13629 raft_consensus.cc:729] T 696778f02edc4bd4a16747ce65f692cc P 61a52b81dbce461eb4a07db7c63fc6e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61a52b81dbce461eb4a07db7c63fc6e5, State: Initialized, Role: FOLLOWER
I20260811 17:19:15.442282 13629 consensus_queue.cc:260] T 696778f02edc4bd4a16747ce65f692cc P 61a52b81dbce461eb4a07db7c63fc6e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:15.442610 13629 ts_tablet_manager.cc:1428] T 696778f02edc4bd4a16747ce65f692cc P 61a52b81dbce461eb4a07db7c63fc6e5: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 17:19:15.442827 13627 tablet_bootstrap.cc:492] T 696778f02edc4bd4a16747ce65f692cc P 70e86f03eaec40a6b498346aad09c6a5: No bootstrap required, opened a new log
I20260811 17:19:15.442945 13627 ts_tablet_manager.cc:1397] T 696778f02edc4bd4a16747ce65f692cc P 70e86f03eaec40a6b498346aad09c6a5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260811 17:19:15.443504 13627 raft_consensus.cc:348] T 696778f02edc4bd4a16747ce65f692cc P 70e86f03eaec40a6b498346aad09c6a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:15.443629 13627 raft_consensus.cc:374] T 696778f02edc4bd4a16747ce65f692cc P 70e86f03eaec40a6b498346aad09c6a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:15.443696 13627 raft_consensus.cc:729] T 696778f02edc4bd4a16747ce65f692cc P 70e86f03eaec40a6b498346aad09c6a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70e86f03eaec40a6b498346aad09c6a5, State: Initialized, Role: FOLLOWER
I20260811 17:19:15.443856 13627 consensus_queue.cc:260] T 696778f02edc4bd4a16747ce65f692cc P 70e86f03eaec40a6b498346aad09c6a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:15.444140 13638 tablet_replica.cc:439] T 696778f02edc4bd4a16747ce65f692cc P 61a52b81dbce461eb4a07db7c63fc6e5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } } }
I20260811 17:19:15.444319 13638 tablet_replica.cc:442] T 696778f02edc4bd4a16747ce65f692cc P 61a52b81dbce461eb4a07db7c63fc6e5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:19:15.444624 13627 ts_tablet_manager.cc:1428] T 696778f02edc4bd4a16747ce65f692cc P 70e86f03eaec40a6b498346aad09c6a5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 17:19:15.444648 13636 tablet_replica.cc:439] T 696778f02edc4bd4a16747ce65f692cc P 70e86f03eaec40a6b498346aad09c6a5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } } }
I20260811 17:19:15.444859 13636 tablet_replica.cc:442] T 696778f02edc4bd4a16747ce65f692cc P 70e86f03eaec40a6b498346aad09c6a5: This TxnStatusTablet replica's current role is: FOLLOWER
W20260811 17:19:15.461064 13595 tablet.cc:2369] T 696778f02edc4bd4a16747ce65f692cc P 61a52b81dbce461eb4a07db7c63fc6e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:19:15.473671 13355 tablet.cc:2369] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:19:15.478945 13127 tablet.cc:2369] T 696778f02edc4bd4a16747ce65f692cc P 70e86f03eaec40a6b498346aad09c6a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:19:15.561805 13639 raft_consensus.cc:482] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:19:15.562019 13639 raft_consensus.cc:504] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:15.563380 13639 leader_election.cc:288] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70e86f03eaec40a6b498346aad09c6a5 (127.9.157.129:39375), 61a52b81dbce461eb4a07db7c63fc6e5 (127.9.157.131:36667)
I20260811 17:19:15.568785 13669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.564205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:36667 (local address 127.9.157.130:52909)
0811 17:19:15.564463 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:15.564477 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:15.564581 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:15.564859 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:15.564861 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:15.564881 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:15.565009 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:15.565015 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.565763 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.565768 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:15.567134 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:15.567147 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.568250 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.568254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:15.568383 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.568412 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 17:19:15.568495 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 17:19:15.568581 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":25,"thread_start_us":96,"threads_started":1}
I20260811 17:19:15.568805 13625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.564463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:52909 (local address 127.9.157.131:36667)
0811 17:19:15.564603 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:15.564610 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:15.564629 (+ 19us) server_negotiation.cc:407] Connection header received
0811 17:19:15.564689 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:15.564693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:15.564747 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:15.564835 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:15.565139 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.565642 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.567284 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.568105 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.568255 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.568333 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:15.568531 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 17:19:15.568639 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 17:19:15.568988 13623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.563900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:55189 (local address 127.9.157.129:39375)
0811 17:19:15.564062 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:15.564073 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:15.564179 (+ 106us) server_negotiation.cc:407] Connection header received
0811 17:19:15.564415 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:15.564418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:15.564473 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:15.564530 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:15.565008 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.565888 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.567790 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.568277 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.568396 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.568447 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:15.568739 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 17:19:15.568830 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 17:19:15.569658 13668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.563723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:39375 (local address 127.9.157.130:55189)
0811 17:19:15.564055 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:15.564077 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:15.564247 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:15.564686 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:15.564689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:15.564705 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:15.564830 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:15.564846 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.566021 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.566025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:15.567158 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:15.567164 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.568509 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.568512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:15.568601 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.568620 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 17:19:15.569457 (+ 837us) client_negotiation.cc:240] Negotiation successful
0811 17:19:15.569547 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":21,"thread_start_us":99,"threads_started":1}
I20260811 17:19:15.569689 13506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "696778f02edc4bd4a16747ce65f692cc" candidate_uuid: "e87af54a41a547d0bc28454d0cee2a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" is_pre_election: true
I20260811 17:19:15.569999 13506 raft_consensus.cc:2454] T 696778f02edc4bd4a16747ce65f692cc P 61a52b81dbce461eb4a07db7c63fc6e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e87af54a41a547d0bc28454d0cee2a06 in term 0.
I20260811 17:19:15.570600 13171 leader_election.cc:302] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61a52b81dbce461eb4a07db7c63fc6e5, e87af54a41a547d0bc28454d0cee2a06; no voters:
I20260811 17:19:15.570724 13036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "696778f02edc4bd4a16747ce65f692cc" candidate_uuid: "e87af54a41a547d0bc28454d0cee2a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70e86f03eaec40a6b498346aad09c6a5" is_pre_election: true
I20260811 17:19:15.570900 13036 raft_consensus.cc:2454] T 696778f02edc4bd4a16747ce65f692cc P 70e86f03eaec40a6b498346aad09c6a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e87af54a41a547d0bc28454d0cee2a06 in term 0.
I20260811 17:19:15.571139 13639 raft_consensus.cc:2781] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:19:15.571218 13639 raft_consensus.cc:482] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:19:15.571264 13639 raft_consensus.cc:3037] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:15.572849 13639 raft_consensus.cc:504] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:15.573379 13639 leader_election.cc:288] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06 [CANDIDATE]: Term 1 election: Requested vote from peers 70e86f03eaec40a6b498346aad09c6a5 (127.9.157.129:39375), 61a52b81dbce461eb4a07db7c63fc6e5 (127.9.157.131:36667)
I20260811 17:19:15.573737 13036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "696778f02edc4bd4a16747ce65f692cc" candidate_uuid: "e87af54a41a547d0bc28454d0cee2a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70e86f03eaec40a6b498346aad09c6a5"
I20260811 17:19:15.573897 13036 raft_consensus.cc:3037] T 696778f02edc4bd4a16747ce65f692cc P 70e86f03eaec40a6b498346aad09c6a5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:15.573828 13506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "696778f02edc4bd4a16747ce65f692cc" candidate_uuid: "e87af54a41a547d0bc28454d0cee2a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61a52b81dbce461eb4a07db7c63fc6e5"
I20260811 17:19:15.573999 13506 raft_consensus.cc:3037] T 696778f02edc4bd4a16747ce65f692cc P 61a52b81dbce461eb4a07db7c63fc6e5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:15.575080 13506 raft_consensus.cc:2454] T 696778f02edc4bd4a16747ce65f692cc P 61a52b81dbce461eb4a07db7c63fc6e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e87af54a41a547d0bc28454d0cee2a06 in term 1.
I20260811 17:19:15.575315 13036 raft_consensus.cc:2454] T 696778f02edc4bd4a16747ce65f692cc P 70e86f03eaec40a6b498346aad09c6a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e87af54a41a547d0bc28454d0cee2a06 in term 1.
I20260811 17:19:15.575448 13171 leader_election.cc:302] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61a52b81dbce461eb4a07db7c63fc6e5, e87af54a41a547d0bc28454d0cee2a06; no voters:
I20260811 17:19:15.575739 13639 raft_consensus.cc:2781] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:15.575982 13639 raft_consensus.cc:686] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06 [term 1 LEADER]: Becoming Leader. State: Replica: e87af54a41a547d0bc28454d0cee2a06, State: Running, Role: LEADER
I20260811 17:19:15.576444 13639 consensus_queue.cc:237] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:15.579512 13672 tablet_replica.cc:439] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06: TxnStatusTablet state changed. Reason: New leader e87af54a41a547d0bc28454d0cee2a06. Latest consensus state: current_term: 1 leader_uuid: "e87af54a41a547d0bc28454d0cee2a06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } } }
I20260811 17:19:15.579772 13672 tablet_replica.cc:442] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06: This TxnStatusTablet replica's current role is: LEADER
I20260811 17:19:15.580206 13674 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 17:19:15.580457 13674 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 17:19:15.581284 12771 catalog_manager.cc:5495] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06 reported cstate change: term changed from 0 to 1, leader changed from <none> to e87af54a41a547d0bc28454d0cee2a06 (127.9.157.130). New cstate: current_term: 1 leader_uuid: "e87af54a41a547d0bc28454d0cee2a06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } health_report { overall_health: UNKNOWN } } }
I20260811 17:19:15.612587 13620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.603874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48504 (local address 127.9.157.130:35635)
0811 17:19:15.604072 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:15.604077 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:15.604099 (+ 22us) server_negotiation.cc:407] Connection header received
0811 17:19:15.604222 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:15.604226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:15.604305 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:15.604453 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:19:15.608239 (+ 3786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.609039 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.610504 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.610864 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.610975 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.611424 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:19:15.612103 (+ 679us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:19:15.612225 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:15.612340 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 17:19:15.612420 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 17:19:15.612676 13657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.594974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:35635 (local address 127.0.0.1:48504)
0811 17:19:15.596367 (+ 1393us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:15.596389 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:15.596541 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:15.607836 (+ 11295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:15.607840 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:15.607858 (+ 18us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 17:19:15.608035 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:15.608044 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.609193 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.609200 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:15.610318 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:15.610348 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.610996 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.611002 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:15.611127 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.611224 (+ 97us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 17:19:15.612205 (+ 981us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 17:19:15.612231 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 17:19:15.612330 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 17:19:15.612473 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1218}
I20260811 17:19:15.619498 13036 raft_consensus.cc:1261] T 696778f02edc4bd4a16747ce65f692cc P 70e86f03eaec40a6b498346aad09c6a5 [term 1 FOLLOWER]: Refusing update from remote peer e87af54a41a547d0bc28454d0cee2a06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:15.619493 13506 raft_consensus.cc:1261] T 696778f02edc4bd4a16747ce65f692cc P 61a52b81dbce461eb4a07db7c63fc6e5 [term 1 FOLLOWER]: Refusing update from remote peer e87af54a41a547d0bc28454d0cee2a06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:15.620438 13639 consensus_queue.cc:1035] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:15.620756 13672 consensus_queue.cc:1035] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:15.626008 13638 tablet_replica.cc:439] T 696778f02edc4bd4a16747ce65f692cc P 61a52b81dbce461eb4a07db7c63fc6e5: TxnStatusTablet state changed. Reason: New leader e87af54a41a547d0bc28454d0cee2a06. Latest consensus state: current_term: 1 leader_uuid: "e87af54a41a547d0bc28454d0cee2a06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } } }
I20260811 17:19:15.626159 13638 tablet_replica.cc:442] T 696778f02edc4bd4a16747ce65f692cc P 61a52b81dbce461eb4a07db7c63fc6e5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:19:15.626631 13636 tablet_replica.cc:439] T 696778f02edc4bd4a16747ce65f692cc P 70e86f03eaec40a6b498346aad09c6a5: TxnStatusTablet state changed. Reason: New leader e87af54a41a547d0bc28454d0cee2a06. Latest consensus state: current_term: 1 leader_uuid: "e87af54a41a547d0bc28454d0cee2a06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } } }
I20260811 17:19:15.626752 13636 tablet_replica.cc:442] T 696778f02edc4bd4a16747ce65f692cc P 70e86f03eaec40a6b498346aad09c6a5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:19:15.629508 13672 tablet_replica.cc:439] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e87af54a41a547d0bc28454d0cee2a06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } } }
I20260811 17:19:15.629654 13672 tablet_replica.cc:442] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06: This TxnStatusTablet replica's current role is: LEADER
I20260811 17:19:15.631299 13638 tablet_replica.cc:439] T 696778f02edc4bd4a16747ce65f692cc P 61a52b81dbce461eb4a07db7c63fc6e5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e87af54a41a547d0bc28454d0cee2a06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } } }
I20260811 17:19:15.631435 13638 tablet_replica.cc:442] T 696778f02edc4bd4a16747ce65f692cc P 61a52b81dbce461eb4a07db7c63fc6e5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:19:15.631750 13685 mvcc.cc:205] Tried to move back new op lower bound from 7317376023007973376 to 7317376022844882944. Current Snapshot: MvccSnapshot[applied={T|T < 7317376023007973376}]
I20260811 17:19:15.636778 13672 tablet_replica.cc:439] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e87af54a41a547d0bc28454d0cee2a06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } } }
I20260811 17:19:15.636933 13672 tablet_replica.cc:442] T 696778f02edc4bd4a16747ce65f692cc P e87af54a41a547d0bc28454d0cee2a06: This TxnStatusTablet replica's current role is: LEADER
I20260811 17:19:15.639571 13636 tablet_replica.cc:439] T 696778f02edc4bd4a16747ce65f692cc P 70e86f03eaec40a6b498346aad09c6a5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e87af54a41a547d0bc28454d0cee2a06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } } }
I20260811 17:19:15.639751 13636 tablet_replica.cc:442] T 696778f02edc4bd4a16747ce65f692cc P 70e86f03eaec40a6b498346aad09c6a5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:19:15.644819 13625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.637874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53350 (local address 127.9.157.131:36667)
0811 17:19:15.638169 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:15.638174 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:15.638189 (+ 15us) server_negotiation.cc:407] Connection header received
0811 17:19:15.638291 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:15.638294 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:15.638363 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:15.638476 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:19:15.639107 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.640090 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.641613 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.642041 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.642143 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.643210 (+ 1067us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:19:15.643873 (+ 663us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:19:15.644009 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:15.644576 (+ 567us) server_negotiation.cc:299] Negotiation successful
0811 17:19:15.644661 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203}
I20260811 17:19:15.645166 13684 mvcc.cc:205] Tried to move back new op lower bound from 7317376023007973376 to 7317376022844882944. Current Snapshot: MvccSnapshot[applied={T|T < 7317376023007973376}]
I20260811 17:19:15.659803 13604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.654957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:35635 (local address 127.9.157.131:48955)
0811 17:19:15.655217 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:15.655234 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:15.655337 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:15.655689 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:15.655692 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:15.655700 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 17:19:15.655831 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:15.655837 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.656991 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.656995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:15.657971 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:15.657982 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.658671 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.658675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:15.658784 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.658857 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 17:19:15.659414 (+ 557us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 17:19:15.659422 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 17:19:15.659495 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 17:19:15.659597 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148}
I20260811 17:19:15.659808 13620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.655103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:48955 (local address 127.9.157.130:35635)
0811 17:19:15.655254 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:15.655261 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:15.655305 (+ 44us) server_negotiation.cc:407] Connection header received
0811 17:19:15.655461 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:15.655466 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:15.655535 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:15.655622 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:19:15.655972 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.656851 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.658121 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.658526 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.658656 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.659193 (+ 537us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:19:15.659307 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:19:15.659390 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:15.659519 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 17:19:15.659596 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 17:19:15.670879 13036 raft_consensus.cc:1261] T 73e92b26cad94cdabe3d25396913be76 P 70e86f03eaec40a6b498346aad09c6a5 [term 1 FOLLOWER]: Refusing update from remote peer 61a52b81dbce461eb4a07db7c63fc6e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:15.671545 13643 consensus_queue.cc:1035] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:15.672557 13266 raft_consensus.cc:1261] T 73e92b26cad94cdabe3d25396913be76 P e87af54a41a547d0bc28454d0cee2a06 [term 1 FOLLOWER]: Refusing update from remote peer 61a52b81dbce461eb4a07db7c63fc6e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:15.673182 13643 consensus_queue.cc:1035] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:15.680701 13683 mvcc.cc:205] Tried to move back new op lower bound from 7317376023221661696 to 7317376021935194112. Current Snapshot: MvccSnapshot[applied={T|T < 7317376023221661696}]
I20260811 17:19:15.698822 13625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.694278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:41357 (local address 127.9.157.131:36667)
0811 17:19:15.694498 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:15.694504 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:15.694521 (+ 17us) server_negotiation.cc:407] Connection header received
0811 17:19:15.694571 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:15.694576 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:15.694636 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:15.694731 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:19:15.695163 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.695987 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.697490 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.697828 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.697941 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.698275 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:19:15.698380 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:19:15.698453 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:15.698620 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 17:19:15.698681 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260811 17:19:15.699148 13704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.693889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:36667 (local address 127.9.157.130:41357)
0811 17:19:15.694241 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:15.694259 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:15.694384 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:15.694745 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:15.694747 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:15.694756 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 17:19:15.694891 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:15.694899 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.696090 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.696095 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:15.696999 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:15.697011 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.697963 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.697967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:15.698063 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.698116 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 17:19:15.698499 (+ 383us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 17:19:15.698509 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 17:19:15.698572 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 17:19:15.698676 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":113,"threads_started":1}
I20260811 17:19:15.744719 12771 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:19:15.744951 12771 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:19:15.748442 12771 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b4346e5ce39a460589fbe00e6188619f: Sending DeleteTablet for 3 replicas of tablet 73e92b26cad94cdabe3d25396913be76
I20260811 17:19:15.750088 13254 tablet_service.cc:1514] Processing DeleteTablet for tablet 73e92b26cad94cdabe3d25396913be76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:15 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:54808
I20260811 17:19:15.750094 13474 tablet_service.cc:1514] Processing DeleteTablet for tablet 73e92b26cad94cdabe3d25396913be76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:15 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:55686
I20260811 17:19:15.750643 13000 tablet_service.cc:1514] Processing DeleteTablet for tablet 73e92b26cad94cdabe3d25396913be76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:15 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:50316
I20260811 17:19:15.751579 13715 tablet_replica.cc:330] T 73e92b26cad94cdabe3d25396913be76 P 70e86f03eaec40a6b498346aad09c6a5: stopping tablet replica
I20260811 17:19:15.751981 13715 raft_consensus.cc:2229] T 73e92b26cad94cdabe3d25396913be76 P 70e86f03eaec40a6b498346aad09c6a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:19:15.752221 13715 raft_consensus.cc:2258] T 73e92b26cad94cdabe3d25396913be76 P 70e86f03eaec40a6b498346aad09c6a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:15.752151 12771 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48778:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:19:15.752628 12771 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:19:15.756693 13715 ts_tablet_manager.cc:1905] T 73e92b26cad94cdabe3d25396913be76 P 70e86f03eaec40a6b498346aad09c6a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:15.757076 13710 tablet_replica.cc:330] T 73e92b26cad94cdabe3d25396913be76 P e87af54a41a547d0bc28454d0cee2a06: stopping tablet replica
I20260811 17:19:15.758232 13710 raft_consensus.cc:2229] T 73e92b26cad94cdabe3d25396913be76 P e87af54a41a547d0bc28454d0cee2a06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:19:15.758702 13474 tablet_service.cc:1467] Processing CreateTablet for tablet 29ccb573c45041598fb53e7b11373b79 (DEFAULT_TABLE table=test-table [id=42a03911724a45198cc6efd64b9a43e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:15.758818 13000 tablet_service.cc:1467] Processing CreateTablet for tablet 29ccb573c45041598fb53e7b11373b79 (DEFAULT_TABLE table=test-table [id=42a03911724a45198cc6efd64b9a43e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:15.759044 13000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29ccb573c45041598fb53e7b11373b79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:15.759047 13474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29ccb573c45041598fb53e7b11373b79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:15.761760 13627 tablet_bootstrap.cc:492] T 29ccb573c45041598fb53e7b11373b79 P 70e86f03eaec40a6b498346aad09c6a5: Bootstrap starting.
I20260811 17:19:15.762784 13627 tablet_bootstrap.cc:654] T 29ccb573c45041598fb53e7b11373b79 P 70e86f03eaec40a6b498346aad09c6a5: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:15.763804 13629 tablet_bootstrap.cc:492] T 29ccb573c45041598fb53e7b11373b79 P 61a52b81dbce461eb4a07db7c63fc6e5: Bootstrap starting.
I20260811 17:19:15.763782 13715 ts_tablet_manager.cc:1918] T 73e92b26cad94cdabe3d25396913be76 P 70e86f03eaec40a6b498346aad09c6a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:19:15.763937 13715 log.cc:1196] T 73e92b26cad94cdabe3d25396913be76 P 70e86f03eaec40a6b498346aad09c6a5: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-0/wal/wals/73e92b26cad94cdabe3d25396913be76
I20260811 17:19:15.764477 13715 ts_tablet_manager.cc:1939] T 73e92b26cad94cdabe3d25396913be76 P 70e86f03eaec40a6b498346aad09c6a5: Deleting consensus metadata
I20260811 17:19:15.764869 13629 tablet_bootstrap.cc:654] T 29ccb573c45041598fb53e7b11373b79 P 61a52b81dbce461eb4a07db7c63fc6e5: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:15.765879 12750 catalog_manager.cc:4841] TS 70e86f03eaec40a6b498346aad09c6a5 (127.9.157.129:39375): tablet 73e92b26cad94cdabe3d25396913be76 (table test-table [id=f935ed3bc56943fd89632a9521aeeff4]) successfully deleted
I20260811 17:19:15.761744 13254 tablet_service.cc:1467] Processing CreateTablet for tablet 29ccb573c45041598fb53e7b11373b79 (DEFAULT_TABLE table=test-table [id=42a03911724a45198cc6efd64b9a43e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:15.767982 13254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29ccb573c45041598fb53e7b11373b79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:15.770253 13710 raft_consensus.cc:2258] T 73e92b26cad94cdabe3d25396913be76 P e87af54a41a547d0bc28454d0cee2a06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:15.770869 13630 tablet_bootstrap.cc:492] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06: Bootstrap starting.
I20260811 17:19:15.772425 13630 tablet_bootstrap.cc:654] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:15.772699 13713 tablet_replica.cc:330] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5: stopping tablet replica
I20260811 17:19:15.773101 13713 raft_consensus.cc:2229] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:19:15.774209 13713 raft_consensus.cc:2258] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:15.775882 13710 ts_tablet_manager.cc:1905] T 73e92b26cad94cdabe3d25396913be76 P e87af54a41a547d0bc28454d0cee2a06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:15.776484 13627 tablet_bootstrap.cc:492] T 29ccb573c45041598fb53e7b11373b79 P 70e86f03eaec40a6b498346aad09c6a5: No bootstrap required, opened a new log
I20260811 17:19:15.776585 13627 ts_tablet_manager.cc:1397] T 29ccb573c45041598fb53e7b11373b79 P 70e86f03eaec40a6b498346aad09c6a5: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.002s
I20260811 17:19:15.777079 13627 raft_consensus.cc:348] T 29ccb573c45041598fb53e7b11373b79 P 70e86f03eaec40a6b498346aad09c6a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:15.777226 13627 raft_consensus.cc:374] T 29ccb573c45041598fb53e7b11373b79 P 70e86f03eaec40a6b498346aad09c6a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:15.777262 13627 raft_consensus.cc:729] T 29ccb573c45041598fb53e7b11373b79 P 70e86f03eaec40a6b498346aad09c6a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70e86f03eaec40a6b498346aad09c6a5, State: Initialized, Role: FOLLOWER
I20260811 17:19:15.777396 13627 consensus_queue.cc:260] T 29ccb573c45041598fb53e7b11373b79 P 70e86f03eaec40a6b498346aad09c6a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:15.777683 13627 ts_tablet_manager.cc:1428] T 29ccb573c45041598fb53e7b11373b79 P 70e86f03eaec40a6b498346aad09c6a5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 17:19:15.778548 13629 tablet_bootstrap.cc:492] T 29ccb573c45041598fb53e7b11373b79 P 61a52b81dbce461eb4a07db7c63fc6e5: No bootstrap required, opened a new log
I20260811 17:19:15.778677 13629 ts_tablet_manager.cc:1397] T 29ccb573c45041598fb53e7b11373b79 P 61a52b81dbce461eb4a07db7c63fc6e5: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20260811 17:19:15.779139 13629 raft_consensus.cc:348] T 29ccb573c45041598fb53e7b11373b79 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:15.779248 13629 raft_consensus.cc:374] T 29ccb573c45041598fb53e7b11373b79 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:15.779278 13629 raft_consensus.cc:729] T 29ccb573c45041598fb53e7b11373b79 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61a52b81dbce461eb4a07db7c63fc6e5, State: Initialized, Role: FOLLOWER
I20260811 17:19:15.779418 13629 consensus_queue.cc:260] T 29ccb573c45041598fb53e7b11373b79 P 61a52b81dbce461eb4a07db7c63fc6e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:15.779728 13629 ts_tablet_manager.cc:1428] T 29ccb573c45041598fb53e7b11373b79 P 61a52b81dbce461eb4a07db7c63fc6e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 17:19:15.784507 13713 ts_tablet_manager.cc:1905] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:15.784982 13630 tablet_bootstrap.cc:492] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06: No bootstrap required, opened a new log
I20260811 17:19:15.785090 13630 ts_tablet_manager.cc:1397] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20260811 17:19:15.785547 13630 raft_consensus.cc:348] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:15.785709 13630 raft_consensus.cc:374] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:15.785743 13630 raft_consensus.cc:729] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e87af54a41a547d0bc28454d0cee2a06, State: Initialized, Role: FOLLOWER
I20260811 17:19:15.785879 13630 consensus_queue.cc:260] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:15.786144 13630 ts_tablet_manager.cc:1428] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 17:19:15.786689 13710 ts_tablet_manager.cc:1918] T 73e92b26cad94cdabe3d25396913be76 P e87af54a41a547d0bc28454d0cee2a06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:19:15.786849 13710 log.cc:1196] T 73e92b26cad94cdabe3d25396913be76 P e87af54a41a547d0bc28454d0cee2a06: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-1/wal/wals/73e92b26cad94cdabe3d25396913be76
I20260811 17:19:15.787356 13710 ts_tablet_manager.cc:1939] T 73e92b26cad94cdabe3d25396913be76 P e87af54a41a547d0bc28454d0cee2a06: Deleting consensus metadata
I20260811 17:19:15.788897 12745 catalog_manager.cc:4841] TS e87af54a41a547d0bc28454d0cee2a06 (127.9.157.130:35635): tablet 73e92b26cad94cdabe3d25396913be76 (table test-table [id=f935ed3bc56943fd89632a9521aeeff4]) successfully deleted
I20260811 17:19:15.789896 13713 ts_tablet_manager.cc:1918] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:19:15.790009 13713 log.cc:1196] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-2/wal/wals/73e92b26cad94cdabe3d25396913be76
I20260811 17:19:15.790329 13713 ts_tablet_manager.cc:1939] T 73e92b26cad94cdabe3d25396913be76 P 61a52b81dbce461eb4a07db7c63fc6e5: Deleting consensus metadata
I20260811 17:19:15.791607 12750 catalog_manager.cc:4841] TS 61a52b81dbce461eb4a07db7c63fc6e5 (127.9.157.131:36667): tablet 73e92b26cad94cdabe3d25396913be76 (table test-table [id=f935ed3bc56943fd89632a9521aeeff4]) successfully deleted
I20260811 17:19:15.820751 13689 raft_consensus.cc:482] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:19:15.820953 13689 raft_consensus.cc:504] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:15.821430 13689 leader_election.cc:288] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70e86f03eaec40a6b498346aad09c6a5 (127.9.157.129:39375), 61a52b81dbce461eb4a07db7c63fc6e5 (127.9.157.131:36667)
I20260811 17:19:15.821935 13506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29ccb573c45041598fb53e7b11373b79" candidate_uuid: "e87af54a41a547d0bc28454d0cee2a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" is_pre_election: true
I20260811 17:19:15.821950 13035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29ccb573c45041598fb53e7b11373b79" candidate_uuid: "e87af54a41a547d0bc28454d0cee2a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70e86f03eaec40a6b498346aad09c6a5" is_pre_election: true
I20260811 17:19:15.822114 13506 raft_consensus.cc:2454] T 29ccb573c45041598fb53e7b11373b79 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e87af54a41a547d0bc28454d0cee2a06 in term 0.
I20260811 17:19:15.822114 13035 raft_consensus.cc:2454] T 29ccb573c45041598fb53e7b11373b79 P 70e86f03eaec40a6b498346aad09c6a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e87af54a41a547d0bc28454d0cee2a06 in term 0.
I20260811 17:19:15.822477 13171 leader_election.cc:302] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61a52b81dbce461eb4a07db7c63fc6e5, e87af54a41a547d0bc28454d0cee2a06; no voters:
I20260811 17:19:15.822708 13689 raft_consensus.cc:2781] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:19:15.822778 13689 raft_consensus.cc:482] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:19:15.822834 13689 raft_consensus.cc:3037] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:15.824137 13689 raft_consensus.cc:504] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:15.824575 13689 leader_election.cc:288] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06 [CANDIDATE]: Term 1 election: Requested vote from peers 70e86f03eaec40a6b498346aad09c6a5 (127.9.157.129:39375), 61a52b81dbce461eb4a07db7c63fc6e5 (127.9.157.131:36667)
I20260811 17:19:15.825102 13506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29ccb573c45041598fb53e7b11373b79" candidate_uuid: "e87af54a41a547d0bc28454d0cee2a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61a52b81dbce461eb4a07db7c63fc6e5"
I20260811 17:19:15.825088 13035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29ccb573c45041598fb53e7b11373b79" candidate_uuid: "e87af54a41a547d0bc28454d0cee2a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70e86f03eaec40a6b498346aad09c6a5"
I20260811 17:19:15.825242 13506 raft_consensus.cc:3037] T 29ccb573c45041598fb53e7b11373b79 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:15.825242 13035 raft_consensus.cc:3037] T 29ccb573c45041598fb53e7b11373b79 P 70e86f03eaec40a6b498346aad09c6a5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:15.826743 13035 raft_consensus.cc:2454] T 29ccb573c45041598fb53e7b11373b79 P 70e86f03eaec40a6b498346aad09c6a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e87af54a41a547d0bc28454d0cee2a06 in term 1.
I20260811 17:19:15.826743 13506 raft_consensus.cc:2454] T 29ccb573c45041598fb53e7b11373b79 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e87af54a41a547d0bc28454d0cee2a06 in term 1.
I20260811 17:19:15.827160 13171 leader_election.cc:302] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70e86f03eaec40a6b498346aad09c6a5, e87af54a41a547d0bc28454d0cee2a06; no voters:
I20260811 17:19:15.827425 13689 raft_consensus.cc:2781] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:15.827621 13689 raft_consensus.cc:686] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06 [term 1 LEADER]: Becoming Leader. State: Replica: e87af54a41a547d0bc28454d0cee2a06, State: Running, Role: LEADER
I20260811 17:19:15.827811 13689 consensus_queue.cc:237] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:15.829861 12771 catalog_manager.cc:5495] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06 reported cstate change: term changed from 0 to 1, leader changed from <none> to e87af54a41a547d0bc28454d0cee2a06 (127.9.157.130). New cstate: current_term: 1 leader_uuid: "e87af54a41a547d0bc28454d0cee2a06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } health_report { overall_health: UNKNOWN } } }
I20260811 17:19:15.848330 13620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.845095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48516 (local address 127.9.157.130:35635)
0811 17:19:15.845200 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:15.845208 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:15.845260 (+ 52us) server_negotiation.cc:407] Connection header received
0811 17:19:15.845428 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:15.845431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:15.845484 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:15.845567 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:19:15.845839 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.846380 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.847236 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.847467 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.847576 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.847868 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:19:15.847955 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:19:15.848035 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:15.848127 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 17:19:15.848180 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 17:19:15.858019 13704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.854647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:35635 (local address 127.9.157.130:58749)
0811 17:19:15.854774 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:15.854793 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:15.854865 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:15.855113 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:15.855114 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:15.855120 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 17:19:15.855206 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:15.855210 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.856217 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.856219 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:15.856783 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:15.856791 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.857331 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.857333 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:15.857394 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.857432 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 17:19:15.857715 (+ 283us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 17:19:15.857720 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 17:19:15.857770 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 17:19:15.857880 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":25}
I20260811 17:19:15.858067 13620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:15.854704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:58749 (local address 127.9.157.130:35635)
0811 17:19:15.854820 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:15.854824 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:15.854846 (+ 22us) server_negotiation.cc:407] Connection header received
0811 17:19:15.854961 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:15.854964 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:15.855013 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:15.855095 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:19:15.855298 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.856118 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.856911 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:15.857215 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:15.857320 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:15.857547 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:19:15.857634 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:19:15.857705 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:15.857794 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 17:19:15.857878 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":50}
I20260811 17:19:15.864580 13506 raft_consensus.cc:1261] T 29ccb573c45041598fb53e7b11373b79 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 1 FOLLOWER]: Refusing update from remote peer e87af54a41a547d0bc28454d0cee2a06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:15.865088 13689 consensus_queue.cc:1035] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:15.873423 13035 raft_consensus.cc:1261] T 29ccb573c45041598fb53e7b11373b79 P 70e86f03eaec40a6b498346aad09c6a5 [term 1 FOLLOWER]: Refusing update from remote peer e87af54a41a547d0bc28454d0cee2a06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260811 17:19:15.873932 13639 consensus_queue.cc:1035] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 17:19:15.887358 13683 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 17:19:17.458701 12772 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:19:17.458878 12772 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:19:17.461633 12772 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b4346e5ce39a460589fbe00e6188619f: Sending DeleteTablet for 3 replicas of tablet 29ccb573c45041598fb53e7b11373b79
I20260811 17:19:17.462363 13254 tablet_service.cc:1514] Processing DeleteTablet for tablet 29ccb573c45041598fb53e7b11373b79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:17 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:54808
I20260811 17:19:17.462371 13000 tablet_service.cc:1514] Processing DeleteTablet for tablet 29ccb573c45041598fb53e7b11373b79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:17 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:50316
I20260811 17:19:17.462446 13474 tablet_service.cc:1514] Processing DeleteTablet for tablet 29ccb573c45041598fb53e7b11373b79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:17 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:55686
I20260811 17:19:17.462790 13771 tablet_replica.cc:330] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06: stopping tablet replica
I20260811 17:19:17.462958 13771 raft_consensus.cc:2229] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06 [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:19:17.463008 13773 tablet_replica.cc:330] T 29ccb573c45041598fb53e7b11373b79 P 61a52b81dbce461eb4a07db7c63fc6e5: stopping tablet replica
I20260811 17:19:17.463236 13773 raft_consensus.cc:2229] T 29ccb573c45041598fb53e7b11373b79 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:19:17.463374 13773 raft_consensus.cc:2258] T 29ccb573c45041598fb53e7b11373b79 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:17.463420 13771 raft_consensus.cc:2258] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:17.464205 13771 ts_tablet_manager.cc:1905] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:17.464362 13773 ts_tablet_manager.cc:1905] T 29ccb573c45041598fb53e7b11373b79 P 61a52b81dbce461eb4a07db7c63fc6e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:17.466743 13772 tablet_replica.cc:330] T 29ccb573c45041598fb53e7b11373b79 P 70e86f03eaec40a6b498346aad09c6a5: stopping tablet replica
I20260811 17:19:17.466925 13772 raft_consensus.cc:2229] T 29ccb573c45041598fb53e7b11373b79 P 70e86f03eaec40a6b498346aad09c6a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:19:17.467074 13772 raft_consensus.cc:2258] T 29ccb573c45041598fb53e7b11373b79 P 70e86f03eaec40a6b498346aad09c6a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:17.467330 13773 ts_tablet_manager.cc:1918] T 29ccb573c45041598fb53e7b11373b79 P 61a52b81dbce461eb4a07db7c63fc6e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 17:19:17.467337 13771 ts_tablet_manager.cc:1918] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 17:19:17.467417 13771 log.cc:1196] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-1/wal/wals/29ccb573c45041598fb53e7b11373b79
I20260811 17:19:17.467417 13773 log.cc:1196] T 29ccb573c45041598fb53e7b11373b79 P 61a52b81dbce461eb4a07db7c63fc6e5: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-2/wal/wals/29ccb573c45041598fb53e7b11373b79
I20260811 17:19:17.467808 13771 ts_tablet_manager.cc:1939] T 29ccb573c45041598fb53e7b11373b79 P e87af54a41a547d0bc28454d0cee2a06: Deleting consensus metadata
I20260811 17:19:17.467890 13773 ts_tablet_manager.cc:1939] T 29ccb573c45041598fb53e7b11373b79 P 61a52b81dbce461eb4a07db7c63fc6e5: Deleting consensus metadata
I20260811 17:19:17.468389 13772 ts_tablet_manager.cc:1905] T 29ccb573c45041598fb53e7b11373b79 P 70e86f03eaec40a6b498346aad09c6a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:17.468650 12745 catalog_manager.cc:4841] TS e87af54a41a547d0bc28454d0cee2a06 (127.9.157.130:35635): tablet 29ccb573c45041598fb53e7b11373b79 (table test-table [id=42a03911724a45198cc6efd64b9a43e9]) successfully deleted
I20260811 17:19:17.468650 12750 catalog_manager.cc:4841] TS 61a52b81dbce461eb4a07db7c63fc6e5 (127.9.157.131:36667): tablet 29ccb573c45041598fb53e7b11373b79 (table test-table [id=42a03911724a45198cc6efd64b9a43e9]) successfully deleted
I20260811 17:19:17.471585 13772 ts_tablet_manager.cc:1918] T 29ccb573c45041598fb53e7b11373b79 P 70e86f03eaec40a6b498346aad09c6a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 17:19:17.471735 13772 log.cc:1196] T 29ccb573c45041598fb53e7b11373b79 P 70e86f03eaec40a6b498346aad09c6a5: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-0/wal/wals/29ccb573c45041598fb53e7b11373b79
I20260811 17:19:17.472016 13772 ts_tablet_manager.cc:1939] T 29ccb573c45041598fb53e7b11373b79 P 70e86f03eaec40a6b498346aad09c6a5: Deleting consensus metadata
I20260811 17:19:17.472807 12750 catalog_manager.cc:4841] TS 70e86f03eaec40a6b498346aad09c6a5 (127.9.157.129:39375): tablet 29ccb573c45041598fb53e7b11373b79 (table test-table [id=42a03911724a45198cc6efd64b9a43e9]) successfully deleted
Aug 11 17:19:17 dist-test-slave-mjjr krb5kdc[12696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468757, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 17:19:17 dist-test-slave-mjjr krb5kdc[12696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468757, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:17.496450 13784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:17.486668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51772 (local address 127.9.157.190:35395)
0811 17:19:17.487028 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:17.487035 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:17.487127 (+ 92us) server_negotiation.cc:407] Connection header received
0811 17:19:17.487260 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:17.487263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:17.487335 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:17.487460 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:17.488341 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:17.488963 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:17.489847 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:17.490086 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:17.490197 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:17.493029 (+ 2832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:17.493033 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:17.493035 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:17.493064 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:17.494751 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:17.495174 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:17.495177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:17.495178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:17.495258 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:17.495499 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:17.495502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:17.495503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:17.495787 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:17.495914 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:17.496107 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 17:19:17.496221 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":107,"threads_started":1}
I20260811 17:19:17.499493 12772 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51772:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:19:17.500046 12772 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:19:17.505275 13000 tablet_service.cc:1467] Processing CreateTablet for tablet 824a7ec65b3e42df9f993c7b6fba7b45 (DEFAULT_TABLE table=test-table [id=2bf28aa76a65464a8e23f286c814ed8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:17.505383 13474 tablet_service.cc:1467] Processing CreateTablet for tablet 824a7ec65b3e42df9f993c7b6fba7b45 (DEFAULT_TABLE table=test-table [id=2bf28aa76a65464a8e23f286c814ed8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:17.505664 13000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 824a7ec65b3e42df9f993c7b6fba7b45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:17.505601 13254 tablet_service.cc:1467] Processing CreateTablet for tablet 824a7ec65b3e42df9f993c7b6fba7b45 (DEFAULT_TABLE table=test-table [id=2bf28aa76a65464a8e23f286c814ed8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:17.505694 13474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 824a7ec65b3e42df9f993c7b6fba7b45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:17.505903 13254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 824a7ec65b3e42df9f993c7b6fba7b45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:17.508471 13786 tablet_bootstrap.cc:492] T 824a7ec65b3e42df9f993c7b6fba7b45 P 70e86f03eaec40a6b498346aad09c6a5: Bootstrap starting.
I20260811 17:19:17.508714 13788 tablet_bootstrap.cc:492] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06: Bootstrap starting.
I20260811 17:19:17.508800 13787 tablet_bootstrap.cc:492] T 824a7ec65b3e42df9f993c7b6fba7b45 P 61a52b81dbce461eb4a07db7c63fc6e5: Bootstrap starting.
I20260811 17:19:17.509579 13787 tablet_bootstrap.cc:654] T 824a7ec65b3e42df9f993c7b6fba7b45 P 61a52b81dbce461eb4a07db7c63fc6e5: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:17.509583 13788 tablet_bootstrap.cc:654] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:17.509583 13786 tablet_bootstrap.cc:654] T 824a7ec65b3e42df9f993c7b6fba7b45 P 70e86f03eaec40a6b498346aad09c6a5: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:17.511015 13787 tablet_bootstrap.cc:492] T 824a7ec65b3e42df9f993c7b6fba7b45 P 61a52b81dbce461eb4a07db7c63fc6e5: No bootstrap required, opened a new log
I20260811 17:19:17.511132 13787 ts_tablet_manager.cc:1397] T 824a7ec65b3e42df9f993c7b6fba7b45 P 61a52b81dbce461eb4a07db7c63fc6e5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 17:19:17.511206 13786 tablet_bootstrap.cc:492] T 824a7ec65b3e42df9f993c7b6fba7b45 P 70e86f03eaec40a6b498346aad09c6a5: No bootstrap required, opened a new log
I20260811 17:19:17.511297 13786 ts_tablet_manager.cc:1397] T 824a7ec65b3e42df9f993c7b6fba7b45 P 70e86f03eaec40a6b498346aad09c6a5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 17:19:17.511677 13788 tablet_bootstrap.cc:492] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06: No bootstrap required, opened a new log
I20260811 17:19:17.511773 13788 ts_tablet_manager.cc:1397] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 17:19:17.511631 13787 raft_consensus.cc:348] T 824a7ec65b3e42df9f993c7b6fba7b45 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:17.511826 13787 raft_consensus.cc:374] T 824a7ec65b3e42df9f993c7b6fba7b45 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:17.511875 13787 raft_consensus.cc:729] T 824a7ec65b3e42df9f993c7b6fba7b45 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61a52b81dbce461eb4a07db7c63fc6e5, State: Initialized, Role: FOLLOWER
I20260811 17:19:17.511837 13786 raft_consensus.cc:348] T 824a7ec65b3e42df9f993c7b6fba7b45 P 70e86f03eaec40a6b498346aad09c6a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:17.511955 13786 raft_consensus.cc:374] T 824a7ec65b3e42df9f993c7b6fba7b45 P 70e86f03eaec40a6b498346aad09c6a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:17.511994 13786 raft_consensus.cc:729] T 824a7ec65b3e42df9f993c7b6fba7b45 P 70e86f03eaec40a6b498346aad09c6a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70e86f03eaec40a6b498346aad09c6a5, State: Initialized, Role: FOLLOWER
I20260811 17:19:17.512035 13787 consensus_queue.cc:260] T 824a7ec65b3e42df9f993c7b6fba7b45 P 61a52b81dbce461eb4a07db7c63fc6e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:17.512130 13786 consensus_queue.cc:260] T 824a7ec65b3e42df9f993c7b6fba7b45 P 70e86f03eaec40a6b498346aad09c6a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:17.512220 13788 raft_consensus.cc:348] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:17.512356 13788 raft_consensus.cc:374] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:17.512399 13788 raft_consensus.cc:729] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e87af54a41a547d0bc28454d0cee2a06, State: Initialized, Role: FOLLOWER
I20260811 17:19:17.512483 13787 ts_tablet_manager.cc:1428] T 824a7ec65b3e42df9f993c7b6fba7b45 P 61a52b81dbce461eb4a07db7c63fc6e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 17:19:17.512544 13786 ts_tablet_manager.cc:1428] T 824a7ec65b3e42df9f993c7b6fba7b45 P 70e86f03eaec40a6b498346aad09c6a5: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 17:19:17.512535 13788 consensus_queue.cc:260] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:17.512807 13788 ts_tablet_manager.cc:1428] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 17:19:17.660502 13746 raft_consensus.cc:482] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:19:17.660697 13746 raft_consensus.cc:504] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:17.661258 13746 leader_election.cc:288] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70e86f03eaec40a6b498346aad09c6a5 (127.9.157.129:39375), 61a52b81dbce461eb4a07db7c63fc6e5 (127.9.157.131:36667)
I20260811 17:19:17.661998 13035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "824a7ec65b3e42df9f993c7b6fba7b45" candidate_uuid: "e87af54a41a547d0bc28454d0cee2a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70e86f03eaec40a6b498346aad09c6a5" is_pre_election: true
I20260811 17:19:17.662067 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "824a7ec65b3e42df9f993c7b6fba7b45" candidate_uuid: "e87af54a41a547d0bc28454d0cee2a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" is_pre_election: true
I20260811 17:19:17.662173 13035 raft_consensus.cc:2454] T 824a7ec65b3e42df9f993c7b6fba7b45 P 70e86f03eaec40a6b498346aad09c6a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e87af54a41a547d0bc28454d0cee2a06 in term 0.
I20260811 17:19:17.662290 13505 raft_consensus.cc:2454] T 824a7ec65b3e42df9f993c7b6fba7b45 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e87af54a41a547d0bc28454d0cee2a06 in term 0.
I20260811 17:19:17.662609 13171 leader_election.cc:302] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70e86f03eaec40a6b498346aad09c6a5, e87af54a41a547d0bc28454d0cee2a06; no voters:
I20260811 17:19:17.662811 13746 raft_consensus.cc:2781] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:19:17.662879 13746 raft_consensus.cc:482] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:19:17.662927 13746 raft_consensus.cc:3037] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:17.664168 13746 raft_consensus.cc:504] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:17.664530 13746 leader_election.cc:288] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06 [CANDIDATE]: Term 1 election: Requested vote from peers 70e86f03eaec40a6b498346aad09c6a5 (127.9.157.129:39375), 61a52b81dbce461eb4a07db7c63fc6e5 (127.9.157.131:36667)
I20260811 17:19:17.665026 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "824a7ec65b3e42df9f993c7b6fba7b45" candidate_uuid: "e87af54a41a547d0bc28454d0cee2a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61a52b81dbce461eb4a07db7c63fc6e5"
I20260811 17:19:17.665026 13035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "824a7ec65b3e42df9f993c7b6fba7b45" candidate_uuid: "e87af54a41a547d0bc28454d0cee2a06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70e86f03eaec40a6b498346aad09c6a5"
I20260811 17:19:17.665155 13505 raft_consensus.cc:3037] T 824a7ec65b3e42df9f993c7b6fba7b45 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:17.665155 13035 raft_consensus.cc:3037] T 824a7ec65b3e42df9f993c7b6fba7b45 P 70e86f03eaec40a6b498346aad09c6a5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:17.666344 13035 raft_consensus.cc:2454] T 824a7ec65b3e42df9f993c7b6fba7b45 P 70e86f03eaec40a6b498346aad09c6a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e87af54a41a547d0bc28454d0cee2a06 in term 1.
I20260811 17:19:17.666347 13505 raft_consensus.cc:2454] T 824a7ec65b3e42df9f993c7b6fba7b45 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e87af54a41a547d0bc28454d0cee2a06 in term 1.
I20260811 17:19:17.666671 13171 leader_election.cc:302] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70e86f03eaec40a6b498346aad09c6a5, e87af54a41a547d0bc28454d0cee2a06; no voters:
I20260811 17:19:17.666915 13746 raft_consensus.cc:2781] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:17.667037 13746 raft_consensus.cc:686] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06 [term 1 LEADER]: Becoming Leader. State: Replica: e87af54a41a547d0bc28454d0cee2a06, State: Running, Role: LEADER
I20260811 17:19:17.667196 13746 consensus_queue.cc:237] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:17.669265 12773 catalog_manager.cc:5495] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06 reported cstate change: term changed from 0 to 1, leader changed from <none> to e87af54a41a547d0bc28454d0cee2a06 (127.9.157.130). New cstate: current_term: 1 leader_uuid: "e87af54a41a547d0bc28454d0cee2a06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } health_report { overall_health: UNKNOWN } } }
I20260811 17:19:17.716687 13804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:17.713427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48526 (local address 127.9.157.130:35635)
0811 17:19:17.713678 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:17.713685 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:17.713729 (+ 44us) server_negotiation.cc:407] Connection header received
0811 17:19:17.713782 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:17.713784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:17.713826 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:17.713905 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:19:17.714195 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:17.714720 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:17.715611 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:17.715911 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:17.715998 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:17.716264 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:19:17.716351 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:19:17.716406 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:17.716512 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 17:19:17.716560 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":112,"threads_started":1}
I20260811 17:19:17.723369 13505 raft_consensus.cc:1261] T 824a7ec65b3e42df9f993c7b6fba7b45 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 1 FOLLOWER]: Refusing update from remote peer e87af54a41a547d0bc28454d0cee2a06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:17.723914 13760 consensus_queue.cc:1035] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:17.724066 13035 raft_consensus.cc:1261] T 824a7ec65b3e42df9f993c7b6fba7b45 P 70e86f03eaec40a6b498346aad09c6a5 [term 1 FOLLOWER]: Refusing update from remote peer e87af54a41a547d0bc28454d0cee2a06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:17.724608 13760 consensus_queue.cc:1035] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:17.762486 12771 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:19:17.762667 12771 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:19:17.764925 12771 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b4346e5ce39a460589fbe00e6188619f: Sending DeleteTablet for 3 replicas of tablet 824a7ec65b3e42df9f993c7b6fba7b45
I20260811 17:19:17.765543 13000 tablet_service.cc:1514] Processing DeleteTablet for tablet 824a7ec65b3e42df9f993c7b6fba7b45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:17 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:50316
I20260811 17:19:17.765558 13253 tablet_service.cc:1514] Processing DeleteTablet for tablet 824a7ec65b3e42df9f993c7b6fba7b45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:17 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:54808
I20260811 17:19:17.765558 13474 tablet_service.cc:1514] Processing DeleteTablet for tablet 824a7ec65b3e42df9f993c7b6fba7b45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:17 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:55686
I20260811 17:19:17.765738 13772 tablet_replica.cc:330] T 824a7ec65b3e42df9f993c7b6fba7b45 P 70e86f03eaec40a6b498346aad09c6a5: stopping tablet replica
I20260811 17:19:17.765774 13773 tablet_replica.cc:330] T 824a7ec65b3e42df9f993c7b6fba7b45 P 61a52b81dbce461eb4a07db7c63fc6e5: stopping tablet replica
I20260811 17:19:17.765882 13773 raft_consensus.cc:2229] T 824a7ec65b3e42df9f993c7b6fba7b45 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:19:17.765887 13772 raft_consensus.cc:2229] T 824a7ec65b3e42df9f993c7b6fba7b45 P 70e86f03eaec40a6b498346aad09c6a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:19:17.765974 13773 raft_consensus.cc:2258] T 824a7ec65b3e42df9f993c7b6fba7b45 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:17.766016 13772 raft_consensus.cc:2258] T 824a7ec65b3e42df9f993c7b6fba7b45 P 70e86f03eaec40a6b498346aad09c6a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:17.766225 13771 tablet_replica.cc:330] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06: stopping tablet replica
I20260811 17:19:17.766366 13771 raft_consensus.cc:2229] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06 [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:19:17.766289 12771 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51772:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 17:19:17.766588 13771 raft_consensus.cc:2258] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 17:19:17.766786 12771 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:19:17.766888 13773 ts_tablet_manager.cc:1905] T 824a7ec65b3e42df9f993c7b6fba7b45 P 61a52b81dbce461eb4a07db7c63fc6e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:17.767314 13771 ts_tablet_manager.cc:1905] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:17.767356 13772 ts_tablet_manager.cc:1905] T 824a7ec65b3e42df9f993c7b6fba7b45 P 70e86f03eaec40a6b498346aad09c6a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:17.770021 13773 ts_tablet_manager.cc:1918] T 824a7ec65b3e42df9f993c7b6fba7b45 P 61a52b81dbce461eb4a07db7c63fc6e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:19:17.770059 13771 ts_tablet_manager.cc:1918] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:19:17.770128 13773 log.cc:1196] T 824a7ec65b3e42df9f993c7b6fba7b45 P 61a52b81dbce461eb4a07db7c63fc6e5: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-2/wal/wals/824a7ec65b3e42df9f993c7b6fba7b45
I20260811 17:19:17.770153 13771 log.cc:1196] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-1/wal/wals/824a7ec65b3e42df9f993c7b6fba7b45
I20260811 17:19:17.770413 13771 ts_tablet_manager.cc:1939] T 824a7ec65b3e42df9f993c7b6fba7b45 P e87af54a41a547d0bc28454d0cee2a06: Deleting consensus metadata
I20260811 17:19:17.770435 13773 ts_tablet_manager.cc:1939] T 824a7ec65b3e42df9f993c7b6fba7b45 P 61a52b81dbce461eb4a07db7c63fc6e5: Deleting consensus metadata
I20260811 17:19:17.771179 12750 catalog_manager.cc:4841] TS 61a52b81dbce461eb4a07db7c63fc6e5 (127.9.157.131:36667): tablet 824a7ec65b3e42df9f993c7b6fba7b45 (table test-table [id=2bf28aa76a65464a8e23f286c814ed8c]) successfully deleted
I20260811 17:19:17.771387 12745 catalog_manager.cc:4841] TS e87af54a41a547d0bc28454d0cee2a06 (127.9.157.130:35635): tablet 824a7ec65b3e42df9f993c7b6fba7b45 (table test-table [id=2bf28aa76a65464a8e23f286c814ed8c]) successfully deleted
I20260811 17:19:17.773849 13772 ts_tablet_manager.cc:1918] T 824a7ec65b3e42df9f993c7b6fba7b45 P 70e86f03eaec40a6b498346aad09c6a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:19:17.773943 13772 log.cc:1196] T 824a7ec65b3e42df9f993c7b6fba7b45 P 70e86f03eaec40a6b498346aad09c6a5: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-0/wal/wals/824a7ec65b3e42df9f993c7b6fba7b45
I20260811 17:19:17.774206 13772 ts_tablet_manager.cc:1939] T 824a7ec65b3e42df9f993c7b6fba7b45 P 70e86f03eaec40a6b498346aad09c6a5: Deleting consensus metadata
I20260811 17:19:17.775525 12750 catalog_manager.cc:4841] TS 70e86f03eaec40a6b498346aad09c6a5 (127.9.157.129:39375): tablet 824a7ec65b3e42df9f993c7b6fba7b45 (table test-table [id=2bf28aa76a65464a8e23f286c814ed8c]) successfully deleted
I20260811 17:19:17.775890 13000 tablet_service.cc:1467] Processing CreateTablet for tablet 90fbcd3d369843cfa22c7d120cc893b8 (DEFAULT_TABLE table=test-table [id=9412d3101d374d76bb606d1d05c63412]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:17.775892 13253 tablet_service.cc:1467] Processing CreateTablet for tablet 90fbcd3d369843cfa22c7d120cc893b8 (DEFAULT_TABLE table=test-table [id=9412d3101d374d76bb606d1d05c63412]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:17.776160 13000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90fbcd3d369843cfa22c7d120cc893b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:17.776160 13253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90fbcd3d369843cfa22c7d120cc893b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:17.776154 13474 tablet_service.cc:1467] Processing CreateTablet for tablet 90fbcd3d369843cfa22c7d120cc893b8 (DEFAULT_TABLE table=test-table [id=9412d3101d374d76bb606d1d05c63412]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:17.776441 13474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90fbcd3d369843cfa22c7d120cc893b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:17.778563 13786 tablet_bootstrap.cc:492] T 90fbcd3d369843cfa22c7d120cc893b8 P 70e86f03eaec40a6b498346aad09c6a5: Bootstrap starting.
I20260811 17:19:17.778650 13787 tablet_bootstrap.cc:492] T 90fbcd3d369843cfa22c7d120cc893b8 P 61a52b81dbce461eb4a07db7c63fc6e5: Bootstrap starting.
I20260811 17:19:17.778683 13788 tablet_bootstrap.cc:492] T 90fbcd3d369843cfa22c7d120cc893b8 P e87af54a41a547d0bc28454d0cee2a06: Bootstrap starting.
I20260811 17:19:17.779516 13786 tablet_bootstrap.cc:654] T 90fbcd3d369843cfa22c7d120cc893b8 P 70e86f03eaec40a6b498346aad09c6a5: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:17.779516 13788 tablet_bootstrap.cc:654] T 90fbcd3d369843cfa22c7d120cc893b8 P e87af54a41a547d0bc28454d0cee2a06: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:17.779556 13787 tablet_bootstrap.cc:654] T 90fbcd3d369843cfa22c7d120cc893b8 P 61a52b81dbce461eb4a07db7c63fc6e5: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:17.780632 13787 tablet_bootstrap.cc:492] T 90fbcd3d369843cfa22c7d120cc893b8 P 61a52b81dbce461eb4a07db7c63fc6e5: No bootstrap required, opened a new log
I20260811 17:19:17.780745 13787 ts_tablet_manager.cc:1397] T 90fbcd3d369843cfa22c7d120cc893b8 P 61a52b81dbce461eb4a07db7c63fc6e5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 17:19:17.781097 13788 tablet_bootstrap.cc:492] T 90fbcd3d369843cfa22c7d120cc893b8 P e87af54a41a547d0bc28454d0cee2a06: No bootstrap required, opened a new log
I20260811 17:19:17.781189 13788 ts_tablet_manager.cc:1397] T 90fbcd3d369843cfa22c7d120cc893b8 P e87af54a41a547d0bc28454d0cee2a06: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 17:19:17.781152 13787 raft_consensus.cc:348] T 90fbcd3d369843cfa22c7d120cc893b8 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:17.781272 13787 raft_consensus.cc:374] T 90fbcd3d369843cfa22c7d120cc893b8 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:17.781312 13787 raft_consensus.cc:729] T 90fbcd3d369843cfa22c7d120cc893b8 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61a52b81dbce461eb4a07db7c63fc6e5, State: Initialized, Role: FOLLOWER
I20260811 17:19:17.781454 13787 consensus_queue.cc:260] T 90fbcd3d369843cfa22c7d120cc893b8 P 61a52b81dbce461eb4a07db7c63fc6e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:17.781553 13786 tablet_bootstrap.cc:492] T 90fbcd3d369843cfa22c7d120cc893b8 P 70e86f03eaec40a6b498346aad09c6a5: No bootstrap required, opened a new log
I20260811 17:19:17.781627 13786 ts_tablet_manager.cc:1397] T 90fbcd3d369843cfa22c7d120cc893b8 P 70e86f03eaec40a6b498346aad09c6a5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 17:19:17.781602 13788 raft_consensus.cc:348] T 90fbcd3d369843cfa22c7d120cc893b8 P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:17.781724 13788 raft_consensus.cc:374] T 90fbcd3d369843cfa22c7d120cc893b8 P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:17.781749 13787 ts_tablet_manager.cc:1428] T 90fbcd3d369843cfa22c7d120cc893b8 P 61a52b81dbce461eb4a07db7c63fc6e5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 17:19:17.781780 13788 raft_consensus.cc:729] T 90fbcd3d369843cfa22c7d120cc893b8 P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e87af54a41a547d0bc28454d0cee2a06, State: Initialized, Role: FOLLOWER
I20260811 17:19:17.781898 13788 consensus_queue.cc:260] T 90fbcd3d369843cfa22c7d120cc893b8 P e87af54a41a547d0bc28454d0cee2a06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:17.782143 13788 ts_tablet_manager.cc:1428] T 90fbcd3d369843cfa22c7d120cc893b8 P e87af54a41a547d0bc28454d0cee2a06: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 17:19:17.782099 13786 raft_consensus.cc:348] T 90fbcd3d369843cfa22c7d120cc893b8 P 70e86f03eaec40a6b498346aad09c6a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:17.782197 13786 raft_consensus.cc:374] T 90fbcd3d369843cfa22c7d120cc893b8 P 70e86f03eaec40a6b498346aad09c6a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:17.782249 13786 raft_consensus.cc:729] T 90fbcd3d369843cfa22c7d120cc893b8 P 70e86f03eaec40a6b498346aad09c6a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70e86f03eaec40a6b498346aad09c6a5, State: Initialized, Role: FOLLOWER
I20260811 17:19:17.782366 13786 consensus_queue.cc:260] T 90fbcd3d369843cfa22c7d120cc893b8 P 70e86f03eaec40a6b498346aad09c6a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:17.782634 13786 ts_tablet_manager.cc:1428] T 90fbcd3d369843cfa22c7d120cc893b8 P 70e86f03eaec40a6b498346aad09c6a5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 17:19:17.830080 13800 raft_consensus.cc:482] T 90fbcd3d369843cfa22c7d120cc893b8 P 70e86f03eaec40a6b498346aad09c6a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:19:17.830266 13800 raft_consensus.cc:504] T 90fbcd3d369843cfa22c7d120cc893b8 P 70e86f03eaec40a6b498346aad09c6a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:17.831523 13800 leader_election.cc:288] T 90fbcd3d369843cfa22c7d120cc893b8 P 70e86f03eaec40a6b498346aad09c6a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e87af54a41a547d0bc28454d0cee2a06 (127.9.157.130:35635), 61a52b81dbce461eb4a07db7c63fc6e5 (127.9.157.131:36667)
I20260811 17:19:17.835938 13804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:17.832202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:51601 (local address 127.9.157.130:35635)
0811 17:19:17.832345 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:17.832350 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:17.832365 (+ 15us) server_negotiation.cc:407] Connection header received
0811 17:19:17.832475 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:17.832478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:17.832530 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:17.832618 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:17.832999 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:17.833571 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:17.834870 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:17.835269 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:17.835403 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:17.835465 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:17.835733 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 17:19:17.835823 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 17:19:17.835938 13818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:17.832027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:39685 (local address 127.9.157.131:36667)
0811 17:19:17.832421 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:17.832437 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:17.832452 (+ 15us) server_negotiation.cc:407] Connection header received
0811 17:19:17.832513 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:17.832515 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:17.832554 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:17.832655 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:17.833018 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:17.833622 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:17.834879 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:17.835280 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:17.835414 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:17.835474 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:17.835696 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 17:19:17.835775 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":92,"threads_started":1}
I20260811 17:19:17.835979 13816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:17.831826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:35635 (local address 127.9.157.129:51601)
0811 17:19:17.832243 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:17.832260 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:17.832381 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:17.832643 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:17.832646 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:17.832662 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:17.832834 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:17.832840 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:17.833688 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:17.833693 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:17.834738 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:17.834748 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:17.835399 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:17.835402 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:17.835512 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:17.835582 (+ 70us) client_negotiation.cc:769] Sending connection context
0811 17:19:17.835709 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 17:19:17.835786 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":81,"spinlock_wait_cycles":13056,"thread_start_us":149,"threads_started":1}
I20260811 17:19:17.836181 13815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:17.831826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:36667 (local address 127.9.157.129:39685)
0811 17:19:17.832211 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:17.832228 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:17.832351 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:17.832663 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:17.832666 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:17.832686 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:17.832869 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:17.832875 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:17.833757 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:17.833760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:17.834746 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:17.834755 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:17.835412 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:17.835416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:17.835549 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:17.835567 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 17:19:17.835676 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 17:19:17.835750 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":57,"thread_start_us":91,"threads_started":1}
I20260811 17:19:17.836513 13266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90fbcd3d369843cfa22c7d120cc893b8" candidate_uuid: "70e86f03eaec40a6b498346aad09c6a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e87af54a41a547d0bc28454d0cee2a06" is_pre_election: true
I20260811 17:19:17.836601 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90fbcd3d369843cfa22c7d120cc893b8" candidate_uuid: "70e86f03eaec40a6b498346aad09c6a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" is_pre_election: true
I20260811 17:19:17.836686 13266 raft_consensus.cc:2454] T 90fbcd3d369843cfa22c7d120cc893b8 P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70e86f03eaec40a6b498346aad09c6a5 in term 0.
I20260811 17:19:17.836720 13505 raft_consensus.cc:2454] T 90fbcd3d369843cfa22c7d120cc893b8 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70e86f03eaec40a6b498346aad09c6a5 in term 0.
I20260811 17:19:17.837105 12924 leader_election.cc:302] T 90fbcd3d369843cfa22c7d120cc893b8 P 70e86f03eaec40a6b498346aad09c6a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61a52b81dbce461eb4a07db7c63fc6e5, 70e86f03eaec40a6b498346aad09c6a5; no voters:
I20260811 17:19:17.837369 13800 raft_consensus.cc:2781] T 90fbcd3d369843cfa22c7d120cc893b8 P 70e86f03eaec40a6b498346aad09c6a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:19:17.837435 13800 raft_consensus.cc:482] T 90fbcd3d369843cfa22c7d120cc893b8 P 70e86f03eaec40a6b498346aad09c6a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:19:17.837473 13800 raft_consensus.cc:3037] T 90fbcd3d369843cfa22c7d120cc893b8 P 70e86f03eaec40a6b498346aad09c6a5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:17.838526 13800 raft_consensus.cc:504] T 90fbcd3d369843cfa22c7d120cc893b8 P 70e86f03eaec40a6b498346aad09c6a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:17.838907 13800 leader_election.cc:288] T 90fbcd3d369843cfa22c7d120cc893b8 P 70e86f03eaec40a6b498346aad09c6a5 [CANDIDATE]: Term 1 election: Requested vote from peers e87af54a41a547d0bc28454d0cee2a06 (127.9.157.130:35635), 61a52b81dbce461eb4a07db7c63fc6e5 (127.9.157.131:36667)
I20260811 17:19:17.839234 13266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90fbcd3d369843cfa22c7d120cc893b8" candidate_uuid: "70e86f03eaec40a6b498346aad09c6a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e87af54a41a547d0bc28454d0cee2a06"
I20260811 17:19:17.839360 13266 raft_consensus.cc:3037] T 90fbcd3d369843cfa22c7d120cc893b8 P e87af54a41a547d0bc28454d0cee2a06 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:17.839351 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90fbcd3d369843cfa22c7d120cc893b8" candidate_uuid: "70e86f03eaec40a6b498346aad09c6a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61a52b81dbce461eb4a07db7c63fc6e5"
I20260811 17:19:17.839493 13505 raft_consensus.cc:3037] T 90fbcd3d369843cfa22c7d120cc893b8 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:17.840412 13266 raft_consensus.cc:2454] T 90fbcd3d369843cfa22c7d120cc893b8 P e87af54a41a547d0bc28454d0cee2a06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70e86f03eaec40a6b498346aad09c6a5 in term 1.
I20260811 17:19:17.840583 13505 raft_consensus.cc:2454] T 90fbcd3d369843cfa22c7d120cc893b8 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70e86f03eaec40a6b498346aad09c6a5 in term 1.
I20260811 17:19:17.840760 12921 leader_election.cc:302] T 90fbcd3d369843cfa22c7d120cc893b8 P 70e86f03eaec40a6b498346aad09c6a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70e86f03eaec40a6b498346aad09c6a5, e87af54a41a547d0bc28454d0cee2a06; no voters:
I20260811 17:19:17.840974 13800 raft_consensus.cc:2781] T 90fbcd3d369843cfa22c7d120cc893b8 P 70e86f03eaec40a6b498346aad09c6a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:17.841233 13800 raft_consensus.cc:686] T 90fbcd3d369843cfa22c7d120cc893b8 P 70e86f03eaec40a6b498346aad09c6a5 [term 1 LEADER]: Becoming Leader. State: Replica: 70e86f03eaec40a6b498346aad09c6a5, State: Running, Role: LEADER
I20260811 17:19:17.841514 13800 consensus_queue.cc:237] T 90fbcd3d369843cfa22c7d120cc893b8 P 70e86f03eaec40a6b498346aad09c6a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } }
I20260811 17:19:17.844103 12773 catalog_manager.cc:5495] T 90fbcd3d369843cfa22c7d120cc893b8 P 70e86f03eaec40a6b498346aad09c6a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 70e86f03eaec40a6b498346aad09c6a5 (127.9.157.129). New cstate: current_term: 1 leader_uuid: "70e86f03eaec40a6b498346aad09c6a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e86f03eaec40a6b498346aad09c6a5" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } health_report { overall_health: UNKNOWN } } }
I20260811 17:19:17.876678 13824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:17.871738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60714 (local address 127.9.157.129:39375)
0811 17:19:17.872381 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:17.872391 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:17.872408 (+ 17us) server_negotiation.cc:407] Connection header received
0811 17:19:17.872509 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:17.872512 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:17.872595 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:17.872741 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:19:17.873068 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:17.873836 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:17.874955 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:17.875218 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:17.875338 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:17.875705 (+ 367us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:19:17.876222 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:19:17.876360 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:17.876434 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 17:19:17.876496 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":157,"threads_started":1}
I20260811 17:19:17.888176 13804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:17.884907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:46419 (local address 127.9.157.130:35635)
0811 17:19:17.885048 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:17.885053 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:17.885183 (+ 130us) server_negotiation.cc:407] Connection header received
0811 17:19:17.885314 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:17.885316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:17.885375 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:17.885480 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:19:17.885734 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:17.886263 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:17.887111 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:17.887370 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:17.887451 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:17.887766 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:19:17.887839 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:19:17.887929 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:17.888007 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 17:19:17.888057 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 17:19:17.888193 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:17.884722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:35635 (local address 127.9.157.129:46419)
0811 17:19:17.885080 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:17.885094 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:17.885193 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:17.885497 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:17.885499 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:17.885506 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 17:19:17.885625 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:17.885630 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:17.886359 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:17.886361 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:17.887011 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:17.887020 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:17.887471 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:17.887473 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:17.887544 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:17.887593 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 17:19:17.887940 (+ 347us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 17:19:17.887945 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 17:19:17.887997 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 17:19:17.888060 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":132,"threads_started":1}
I20260811 17:19:17.894767 13505 raft_consensus.cc:1261] T 90fbcd3d369843cfa22c7d120cc893b8 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 1 FOLLOWER]: Refusing update from remote peer 70e86f03eaec40a6b498346aad09c6a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:17.894925 13266 raft_consensus.cc:1261] T 90fbcd3d369843cfa22c7d120cc893b8 P e87af54a41a547d0bc28454d0cee2a06 [term 1 FOLLOWER]: Refusing update from remote peer 70e86f03eaec40a6b498346aad09c6a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:17.895295 13800 consensus_queue.cc:1035] T 90fbcd3d369843cfa22c7d120cc893b8 P 70e86f03eaec40a6b498346aad09c6a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61a52b81dbce461eb4a07db7c63fc6e5" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:17.895573 13822 consensus_queue.cc:1035] T 90fbcd3d369843cfa22c7d120cc893b8 P 70e86f03eaec40a6b498346aad09c6a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e87af54a41a547d0bc28454d0cee2a06" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:17.914108 13840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:17.910341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:39375 (local address 127.9.157.130:34121)
0811 17:19:17.910679 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:17.910695 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:17.910817 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:17.911115 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:17.911120 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:17.911131 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 17:19:17.911255 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:17.911263 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:17.912066 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:17.912069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:17.912791 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:17.912803 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:17.913321 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:17.913325 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:17.913419 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:17.913456 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 17:19:17.913804 (+ 348us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 17:19:17.913813 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 17:19:17.913887 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 17:19:17.913981 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":125,"threads_started":1}
I20260811 17:19:17.914117 13824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:17.910409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:34121 (local address 127.9.157.129:39375)
0811 17:19:17.910669 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:17.910694 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:17.910764 (+ 70us) server_negotiation.cc:407] Connection header received
0811 17:19:17.910944 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:17.910947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:17.911004 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:17.911081 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:19:17.911362 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:17.911959 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:17.912912 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:17.913185 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:17.913319 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:17.913585 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:19:17.913686 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:19:17.913770 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:17.913919 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 17:19:17.913985 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173}
W20260811 17:19:17.915129 13736 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 17:19:19.478773 12771 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:19:19.478950 12771 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:19:19.481602 12771 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b4346e5ce39a460589fbe00e6188619f: Sending DeleteTablet for 3 replicas of tablet 90fbcd3d369843cfa22c7d120cc893b8
I20260811 17:19:19.482236 13000 tablet_service.cc:1514] Processing DeleteTablet for tablet 90fbcd3d369843cfa22c7d120cc893b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:19 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:50316
I20260811 17:19:19.482353 13253 tablet_service.cc:1514] Processing DeleteTablet for tablet 90fbcd3d369843cfa22c7d120cc893b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:19 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:54808
I20260811 17:19:19.482688 13474 tablet_service.cc:1514] Processing DeleteTablet for tablet 90fbcd3d369843cfa22c7d120cc893b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:19 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:55686
I20260811 17:19:19.482831 13893 tablet_replica.cc:330] T 90fbcd3d369843cfa22c7d120cc893b8 P e87af54a41a547d0bc28454d0cee2a06: stopping tablet replica
I20260811 17:19:19.482998 13893 raft_consensus.cc:2229] T 90fbcd3d369843cfa22c7d120cc893b8 P e87af54a41a547d0bc28454d0cee2a06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:19:19.483127 13892 tablet_replica.cc:330] T 90fbcd3d369843cfa22c7d120cc893b8 P 70e86f03eaec40a6b498346aad09c6a5: stopping tablet replica
I20260811 17:19:19.483136 13893 raft_consensus.cc:2258] T 90fbcd3d369843cfa22c7d120cc893b8 P e87af54a41a547d0bc28454d0cee2a06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:19.483515 13892 raft_consensus.cc:2229] T 90fbcd3d369843cfa22c7d120cc893b8 P 70e86f03eaec40a6b498346aad09c6a5 [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:19:19.483937 13892 raft_consensus.cc:2258] T 90fbcd3d369843cfa22c7d120cc893b8 P 70e86f03eaec40a6b498346aad09c6a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:19.484176 13893 ts_tablet_manager.cc:1905] T 90fbcd3d369843cfa22c7d120cc893b8 P e87af54a41a547d0bc28454d0cee2a06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:19.484185 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 12891
I20260811 17:19:19.496127 13893 ts_tablet_manager.cc:1918] T 90fbcd3d369843cfa22c7d120cc893b8 P e87af54a41a547d0bc28454d0cee2a06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 17:19:19.496168 13896 tablet_replica.cc:330] T 90fbcd3d369843cfa22c7d120cc893b8 P 61a52b81dbce461eb4a07db7c63fc6e5: stopping tablet replica
I20260811 17:19:19.496255 13893 log.cc:1196] T 90fbcd3d369843cfa22c7d120cc893b8 P e87af54a41a547d0bc28454d0cee2a06: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-1/wal/wals/90fbcd3d369843cfa22c7d120cc893b8
I20260811 17:19:19.496402 13896 raft_consensus.cc:2229] T 90fbcd3d369843cfa22c7d120cc893b8 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:19:19.496583 13896 raft_consensus.cc:2258] T 90fbcd3d369843cfa22c7d120cc893b8 P 61a52b81dbce461eb4a07db7c63fc6e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:19.496603 13893 ts_tablet_manager.cc:1939] T 90fbcd3d369843cfa22c7d120cc893b8 P e87af54a41a547d0bc28454d0cee2a06: Deleting consensus metadata
I20260811 17:19:19.497381 13896 ts_tablet_manager.cc:1905] T 90fbcd3d369843cfa22c7d120cc893b8 P 61a52b81dbce461eb4a07db7c63fc6e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:19.497617 12745 catalog_manager.cc:4841] TS e87af54a41a547d0bc28454d0cee2a06 (127.9.157.130:35635): tablet 90fbcd3d369843cfa22c7d120cc893b8 (table test-table [id=9412d3101d374d76bb606d1d05c63412]) successfully deleted
I20260811 17:19:19.499775 13896 ts_tablet_manager.cc:1918] T 90fbcd3d369843cfa22c7d120cc893b8 P 61a52b81dbce461eb4a07db7c63fc6e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 17:19:19.499872 13896 log.cc:1196] T 90fbcd3d369843cfa22c7d120cc893b8 P 61a52b81dbce461eb4a07db7c63fc6e5: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786468743878666-9846-0/minicluster-data/ts-2/wal/wals/90fbcd3d369843cfa22c7d120cc893b8
I20260811 17:19:19.500159 13896 ts_tablet_manager.cc:1939] T 90fbcd3d369843cfa22c7d120cc893b8 P 61a52b81dbce461eb4a07db7c63fc6e5: Deleting consensus metadata
I20260811 17:19:19.500938 12750 catalog_manager.cc:4841] TS 61a52b81dbce461eb4a07db7c63fc6e5 (127.9.157.131:36667): tablet 90fbcd3d369843cfa22c7d120cc893b8 (table test-table [id=9412d3101d374d76bb606d1d05c63412]) successfully deleted
W20260811 17:19:19.504961 12750 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.157.129:39375 (error 108)
I20260811 17:19:19.505195 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 13138
I20260811 17:19:19.513827 13898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:19.512828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:39375 (local address 127.0.0.1:60716)
0811 17:19:19.513546 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:19.513661 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.129:39375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":381,"threads_started":1}
W20260811 17:19:19.513974 12750 catalog_manager.cc:4584] TS 70e86f03eaec40a6b498346aad09c6a5 (127.9.157.129:39375): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 90fbcd3d369843cfa22c7d120cc893b8: Network error: Client connection negotiation failed: client connection to 127.9.157.129:39375: connect: Connection refused (error 111)
I20260811 17:19:19.529825 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 13371
I20260811 17:19:19.547374 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 12716
2026-08-11T17:19:19Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6782 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:19:19 dist-test-slave-mjjr krb5kdc[13903](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:19:19 dist-test-slave-mjjr krb5kdc[13903](info): set up 2 sockets
Aug 11 17:19:19 dist-test-slave-mjjr krb5kdc[13903](info): commencing operation
krb5kdc: starting...
W20260811 17:19:20.832054 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:19:20 dist-test-slave-mjjr krb5kdc[13903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:19:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:19:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:19:20.970489 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:46055
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:35529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:46055
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:21.091938 13930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:21.092300 13930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:21.092375 13930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:21.096503 13930 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:19:21.096618 13930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:21.096654 13930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:21.096697 13930 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:19:21.096732 13930 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:19:21.102407 13930 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:35529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:46055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:46055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:21.103729 13930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:21.104887 13930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:21.114807 13937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:21.114904 13930 server_base.cc:1034] running on GCE node
W20260811 17:19:21.115005 13945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:21.114817 13940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:21.115789 13930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:21.117098 13930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:21.118291 13930 hybrid_clock.cc:648] HybridClock initialized: now 1786468761118269 us; error 36 us; skew 500 ppm
Aug 11 17:19:21 dist-test-slave-mjjr krb5kdc[13903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468761, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:21.121483 13930 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:19:21.122466 13930 webserver.cc:460] Webserver started at http://127.9.157.190:40525/ using document root <none> and password file <none>
I20260811 17:19:21.123067 13930 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:21.123133 13930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:21.123337 13930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:21.125664 13930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "f6eb16e1f6a2422094913e327fe81dc0"
format_stamp: "Formatted at 2026-08-11 17:19:21 on dist-test-slave-mjjr"
server_key: "211bd1d709fce43b5cfb576716c348bb"
server_key_iv: "e2b8b288723bad1a616e5c7f1404a68c"
server_key_version: "encryptionkey@0"
I20260811 17:19:21.126235 13930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "f6eb16e1f6a2422094913e327fe81dc0"
format_stamp: "Formatted at 2026-08-11 17:19:21 on dist-test-slave-mjjr"
server_key: "211bd1d709fce43b5cfb576716c348bb"
server_key_iv: "e2b8b288723bad1a616e5c7f1404a68c"
server_key_version: "encryptionkey@0"
I20260811 17:19:21.130487 13930 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 17:19:21.133188 13958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:21.134582 13930 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:19:21.134755 13930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "f6eb16e1f6a2422094913e327fe81dc0"
format_stamp: "Formatted at 2026-08-11 17:19:21 on dist-test-slave-mjjr"
server_key: "211bd1d709fce43b5cfb576716c348bb"
server_key_iv: "e2b8b288723bad1a616e5c7f1404a68c"
server_key_version: "encryptionkey@0"
I20260811 17:19:21.134871 13930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:21.161309 13930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:21.164999 13930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:21.165233 13930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:21.172753 13930 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:46055
I20260811 17:19:21.172843 14055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:46055 every 8 connection(s)
I20260811 17:19:21.173918 13930 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:19:21.177192 14057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:21.179149 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 13930
I20260811 17:19:21.179309 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:19:21.179595 9846 external_mini_cluster.cc:1428] Setting key 0b31fbfd23d6ce1176d17d4d3ce96291
I20260811 17:19:21.185050 14057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6eb16e1f6a2422094913e327fe81dc0: Bootstrap starting.
I20260811 17:19:21.188200 14057 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6eb16e1f6a2422094913e327fe81dc0: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:21.189075 14057 log.cc:824] T 00000000000000000000000000000000 P f6eb16e1f6a2422094913e327fe81dc0: Log is configured to *not* fsync() on all Append() calls
Aug 11 17:19:21 dist-test-slave-mjjr krb5kdc[13903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:21.191603 14057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6eb16e1f6a2422094913e327fe81dc0: No bootstrap required, opened a new log
I20260811 17:19:21.195729 14057 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6eb16e1f6a2422094913e327fe81dc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6eb16e1f6a2422094913e327fe81dc0" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 46055 } }
I20260811 17:19:21.196013 14057 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6eb16e1f6a2422094913e327fe81dc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:21.196058 14064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:21.181299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38776 (local address 127.9.157.190:46055)
0811 17:19:21.181836 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:21.181857 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:21.181908 (+ 51us) server_negotiation.cc:407] Connection header received
0811 17:19:21.182871 (+ 963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:21.182904 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:21.183392 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:21.183863 (+ 471us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:21.184659 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:21.186133 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:21.187203 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:21.187956 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:21.188148 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:21.190943 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:21.190948 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:21.190964 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:21.190992 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:21.192802 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:21.193303 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:21.193308 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:21.193314 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:21.193431 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:21.193749 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:21.193753 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:21.193754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:21.194184 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:21.194425 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:21.194760 (+ 335us) server_negotiation.cc:299] Negotiation successful
0811 17:19:21.195133 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":147,"threads_started":1}
I20260811 17:19:21.196095 14057 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6eb16e1f6a2422094913e327fe81dc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6eb16e1f6a2422094913e327fe81dc0, State: Initialized, Role: FOLLOWER
I20260811 17:19:21.196641 14057 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6eb16e1f6a2422094913e327fe81dc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6eb16e1f6a2422094913e327fe81dc0" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 46055 } }
I20260811 17:19:21.196810 14057 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6eb16e1f6a2422094913e327fe81dc0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:19:21.196877 14057 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6eb16e1f6a2422094913e327fe81dc0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:19:21.196971 14057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6eb16e1f6a2422094913e327fe81dc0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:21.198015 14057 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6eb16e1f6a2422094913e327fe81dc0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6eb16e1f6a2422094913e327fe81dc0" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 46055 } }
I20260811 17:19:21.198410 14057 leader_election.cc:302] T 00000000000000000000000000000000 P f6eb16e1f6a2422094913e327fe81dc0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6eb16e1f6a2422094913e327fe81dc0; no voters:
I20260811 17:19:21.198784 14057 leader_election.cc:288] T 00000000000000000000000000000000 P f6eb16e1f6a2422094913e327fe81dc0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:19:21.198906 14068 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6eb16e1f6a2422094913e327fe81dc0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:21.199164 14068 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6eb16e1f6a2422094913e327fe81dc0 [term 1 LEADER]: Becoming Leader. State: Replica: f6eb16e1f6a2422094913e327fe81dc0, State: Running, Role: LEADER
I20260811 17:19:21.199489 14068 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6eb16e1f6a2422094913e327fe81dc0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6eb16e1f6a2422094913e327fe81dc0" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 46055 } }
I20260811 17:19:21.200126 14057 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6eb16e1f6a2422094913e327fe81dc0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:19:21.201290 14073 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6eb16e1f6a2422094913e327fe81dc0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6eb16e1f6a2422094913e327fe81dc0. Latest consensus state: current_term: 1 leader_uuid: "f6eb16e1f6a2422094913e327fe81dc0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6eb16e1f6a2422094913e327fe81dc0" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 46055 } } }
I20260811 17:19:21.201424 14073 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6eb16e1f6a2422094913e327fe81dc0 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:21.201753 14069 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6eb16e1f6a2422094913e327fe81dc0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6eb16e1f6a2422094913e327fe81dc0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6eb16e1f6a2422094913e327fe81dc0" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 46055 } } }
I20260811 17:19:21.201871 14069 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6eb16e1f6a2422094913e327fe81dc0 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:21.201951 14085 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:19:21.205404 14085 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:19:21.212062 14085 catalog_manager.cc:1347] Generated new cluster ID: adcda3a5bd9c47319053b49b27e9a119
I20260811 17:19:21.212164 14085 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:19:21.229892 14085 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:19:21.232434 14085 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:19:21.239912 14085 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6eb16e1f6a2422094913e327fe81dc0: Generated new TSK 0
I20260811 17:19:21.240845 14085 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
I20260811 17:19:21.282971 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:0
--local_ip_for_outbound_sockets=127.9.157.129
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:46055
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:35529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:21.418507 14103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:21.418833 14103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:21.418884 14103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:21.423744 14103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:21.423825 14103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:21.423962 14103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:19:21.430173 14103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:35529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:46055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:21.431597 14103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:21.432842 14103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:21.443671 14110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:21.443740 14103 server_base.cc:1034] running on GCE node
W20260811 17:19:21.443929 14109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:21.443681 14113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:21.444437 14103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:21.445071 14103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:21.446249 14103 hybrid_clock.cc:648] HybridClock initialized: now 1786468761446233 us; error 32 us; skew 500 ppm
Aug 11 17:19:21 dist-test-slave-mjjr krb5kdc[13903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468761, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:21.449411 14103 init.cc:377] Logged in from keytab as kudu/127.9.157.129@KRBTEST.COM (short username kudu)
I20260811 17:19:21.450347 14103 webserver.cc:460] Webserver started at http://127.9.157.129:38597/ using document root <none> and password file <none>
I20260811 17:19:21.450924 14103 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:21.450997 14103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:21.451234 14103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:21.453517 14103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/data/instance:
uuid: "ae6ae77930fd4f3ba81ca3acceca6915"
format_stamp: "Formatted at 2026-08-11 17:19:21 on dist-test-slave-mjjr"
server_key: "4a8309023f8895d28544103ef8f479dd"
server_key_iv: "b6bf26d12ec8b2b6f8bb6cf6a8b7d3b2"
server_key_version: "encryptionkey@0"
I20260811 17:19:21.454092 14103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance:
uuid: "ae6ae77930fd4f3ba81ca3acceca6915"
format_stamp: "Formatted at 2026-08-11 17:19:21 on dist-test-slave-mjjr"
server_key: "4a8309023f8895d28544103ef8f479dd"
server_key_iv: "b6bf26d12ec8b2b6f8bb6cf6a8b7d3b2"
server_key_version: "encryptionkey@0"
I20260811 17:19:21.458148 14103 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 17:19:21.460788 14133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:21.462172 14103 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:21.462311 14103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "ae6ae77930fd4f3ba81ca3acceca6915"
format_stamp: "Formatted at 2026-08-11 17:19:21 on dist-test-slave-mjjr"
server_key: "4a8309023f8895d28544103ef8f479dd"
server_key_iv: "b6bf26d12ec8b2b6f8bb6cf6a8b7d3b2"
server_key_version: "encryptionkey@0"
I20260811 17:19:21.462466 14103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:21.489970 14103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:21.493680 14103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:21.493918 14103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:21.494614 14103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:21.495579 14103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:21.495685 14103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:21.495770 14103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:21.495805 14103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:21.506058 14103 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:35857
I20260811 17:19:21.506078 14317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:35857 every 8 connection(s)
I20260811 17:19:21.507091 14103 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:19:21.512487 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 14103
I20260811 17:19:21.512588 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance
I20260811 17:19:21.512871 9846 external_mini_cluster.cc:1428] Setting key 60a9232815a2bff8af6e3a14d2de53f7
Aug 11 17:19:21 dist-test-slave-mjjr krb5kdc[13903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468761, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:21.522719 14064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:21.509137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:44653 (local address 127.9.157.190:46055)
0811 17:19:21.509330 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:21.509335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:21.510263 (+ 928us) server_negotiation.cc:407] Connection header received
0811 17:19:21.511588 (+ 1325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:21.511590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:21.511684 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:21.511771 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:21.513171 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:21.513850 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:21.515253 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:21.515710 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:21.515840 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:21.518795 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:21.518798 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:21.518800 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:21.518830 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:21.520417 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:21.521014 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:21.521018 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:21.521020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:21.521113 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:21.521498 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:21.521503 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:21.521504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:21.521711 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:21.521820 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:21.522398 (+ 578us) server_negotiation.cc:299] Negotiation successful
0811 17:19:21.522542 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 17:19:21.523721 14322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:21.509585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:46055 (local address 127.9.157.129:44653)
0811 17:19:21.510085 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:21.510135 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:21.511372 (+ 1237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:21.511953 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:21.511960 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:21.512366 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:21.513005 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:21.513023 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:21.514013 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:21.514018 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:21.515115 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:21.515126 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:21.515908 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:21.515913 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:21.516086 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:21.516860 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:21.516884 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:21.518567 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:21.520541 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:21.520547 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:21.520564 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:21.520891 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:21.521234 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:21.521237 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:21.521238 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:21.521394 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:21.521836 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:21.521845 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:21.522137 (+ 292us) client_negotiation.cc:769] Sending connection context
0811 17:19:21.522347 (+ 210us) client_negotiation.cc:240] Negotiation successful
0811 17:19:21.522718 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":119,"threads_started":1}
WARNING: no policy specified for kudu/127.9.157.130@KRBTEST.COM; defaulting to no policy
I20260811 17:19:21.525219 14319 heartbeater.cc:347] Connected to a master server at 127.9.157.190:46055
I20260811 17:19:21.525483 14319 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:21.526005 14319 heartbeater.cc:511] Master 127.9.157.190:46055 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.130@KRBTEST.COM" created.
I20260811 17:19:21.527994 13985 ts_manager.cc:194] Registered new tserver with Master: ae6ae77930fd4f3ba81ca3acceca6915 (127.9.157.129:35857)
I20260811 17:19:21.529210 13985 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.129@KRBTEST.COM'} at 127.9.157.129:44653
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
I20260811 17:19:21.546492 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:0
--local_ip_for_outbound_sockets=127.9.157.130
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:46055
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:35529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:21.669919 14334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:21.670233 14334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:21.670287 14334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:21.674634 14334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:21.674715 14334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:21.674782 14334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:19:21.681262 14334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:35529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:46055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:21.682554 14334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:21.683821 14334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:21.694761 14348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:21.694794 14342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:21.694842 14346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:21.695017 14334 server_base.cc:1034] running on GCE node
I20260811 17:19:21.695806 14334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:21.696524 14334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:21.697750 14334 hybrid_clock.cc:648] HybridClock initialized: now 1786468761697713 us; error 49 us; skew 500 ppm
Aug 11 17:19:21 dist-test-slave-mjjr krb5kdc[13903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468761, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:21.700901 14334 init.cc:377] Logged in from keytab as kudu/127.9.157.130@KRBTEST.COM (short username kudu)
I20260811 17:19:21.702059 14334 webserver.cc:460] Webserver started at http://127.9.157.130:43877/ using document root <none> and password file <none>
I20260811 17:19:21.702761 14334 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:21.702862 14334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:21.703132 14334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:21.705587 14334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/data/instance:
uuid: "2faa64bdafcd418a8a61c5aece6f4534"
format_stamp: "Formatted at 2026-08-11 17:19:21 on dist-test-slave-mjjr"
server_key: "7b3705cc4a85f077eccc646e6ee97268"
server_key_iv: "088ab971af86651a100dcc22acaeb75e"
server_key_version: "encryptionkey@0"
I20260811 17:19:21.706262 14334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance:
uuid: "2faa64bdafcd418a8a61c5aece6f4534"
format_stamp: "Formatted at 2026-08-11 17:19:21 on dist-test-slave-mjjr"
server_key: "7b3705cc4a85f077eccc646e6ee97268"
server_key_iv: "088ab971af86651a100dcc22acaeb75e"
server_key_version: "encryptionkey@0"
I20260811 17:19:21.710619 14334 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:19:21.713557 14359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:21.715050 14334 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:19:21.715229 14334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "2faa64bdafcd418a8a61c5aece6f4534"
format_stamp: "Formatted at 2026-08-11 17:19:21 on dist-test-slave-mjjr"
server_key: "7b3705cc4a85f077eccc646e6ee97268"
server_key_iv: "088ab971af86651a100dcc22acaeb75e"
server_key_version: "encryptionkey@0"
I20260811 17:19:21.715402 14334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:21.737223 14334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:21.741235 14334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:21.741477 14334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:21.742226 14334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:21.743202 14334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:21.743279 14334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:21.743355 14334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:21.743404 14334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:21.753435 14334 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:34553
I20260811 17:19:21.753454 14579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:34553 every 8 connection(s)
I20260811 17:19:21.754444 14334 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:19:21.755189 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 14334
I20260811 17:19:21.755312 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance
I20260811 17:19:21.755609 9846 external_mini_cluster.cc:1428] Setting key 511d2fe660afda5dc6e64e4444c35842
Aug 11 17:19:21 dist-test-slave-mjjr krb5kdc[13903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468761, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.157.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.131@KRBTEST.COM" created.
I20260811 17:19:21.770326 14064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:21.756591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:50361 (local address 127.9.157.190:46055)
0811 17:19:21.756743 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:21.756748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:21.757742 (+ 994us) server_negotiation.cc:407] Connection header received
0811 17:19:21.759165 (+ 1423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:21.759169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:21.759228 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:21.759311 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:21.760670 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:21.761222 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:21.762738 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:21.762983 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:21.763108 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:21.765916 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:21.765920 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:21.765923 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:21.765962 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:21.767830 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:21.768496 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:21.768500 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:21.768502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:21.768601 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:21.769015 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:21.769019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:21.769021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:21.769266 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:21.769389 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:21.769999 (+ 610us) server_negotiation.cc:299] Negotiation successful
0811 17:19:21.770147 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 17:19:21.771484 14584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:21.757066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:46055 (local address 127.9.157.130:50361)
0811 17:19:21.757587 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:21.757634 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:21.758938 (+ 1304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:21.759506 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:21.759513 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:21.759980 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:21.760515 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:21.760530 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:21.761380 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:21.761382 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:21.762595 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:21.762607 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:21.763161 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:21.763165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:21.763354 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:21.764112 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:21.764131 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:21.765697 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:21.767990 (+ 2293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:21.767998 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:21.768026 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:21.768361 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:21.768712 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:21.768715 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:21.768717 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:21.768902 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:21.769392 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:21.769404 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:21.769727 (+ 323us) client_negotiation.cc:769] Sending connection context
0811 17:19:21.769954 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 17:19:21.770374 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":154,"threads_started":1}
I20260811 17:19:21.773059 14580 heartbeater.cc:347] Connected to a master server at 127.9.157.190:46055
I20260811 17:19:21.773415 14580 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:21.774161 14580 heartbeater.cc:511] Master 127.9.157.190:46055 requested a full tablet report, sending...
I20260811 17:19:21.775568 13985 ts_manager.cc:194] Registered new tserver with Master: 2faa64bdafcd418a8a61c5aece6f4534 (127.9.157.130:34553)
I20260811 17:19:21.776262 13985 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.130@KRBTEST.COM'} at 127.9.157.130:50361
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
I20260811 17:19:21.792308 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:0
--local_ip_for_outbound_sockets=127.9.157.131
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:46055
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:35529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:21.922820 14592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:21.923141 14592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:21.923187 14592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:21.927603 14592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:21.927734 14592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:21.927800 14592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:19:21.933679 14592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:35529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:46055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:21.935020 14592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:21.936252 14592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:21.947281 14607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:21.947361 14592 server_base.cc:1034] running on GCE node
W20260811 17:19:21.947296 14602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:21.947296 14604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:21.948065 14592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:21.948838 14592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:21.950033 14592 hybrid_clock.cc:648] HybridClock initialized: now 1786468761949992 us; error 52 us; skew 500 ppm
Aug 11 17:19:21 dist-test-slave-mjjr krb5kdc[13903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468761, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:21.953563 14592 init.cc:377] Logged in from keytab as kudu/127.9.157.131@KRBTEST.COM (short username kudu)
I20260811 17:19:21.954618 14592 webserver.cc:460] Webserver started at http://127.9.157.131:44075/ using document root <none> and password file <none>
I20260811 17:19:21.955245 14592 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:21.955344 14592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:21.955608 14592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:21.957872 14592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/data/instance:
uuid: "f0f10aa1c67c49bdacfa9171323d15cf"
format_stamp: "Formatted at 2026-08-11 17:19:21 on dist-test-slave-mjjr"
server_key: "8594db94d20ffdf1dd53c0474ab84174"
server_key_iv: "04ca54cc5b25e97fee7d745691b8aea0"
server_key_version: "encryptionkey@0"
I20260811 17:19:21.958577 14592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance:
uuid: "f0f10aa1c67c49bdacfa9171323d15cf"
format_stamp: "Formatted at 2026-08-11 17:19:21 on dist-test-slave-mjjr"
server_key: "8594db94d20ffdf1dd53c0474ab84174"
server_key_iv: "04ca54cc5b25e97fee7d745691b8aea0"
server_key_version: "encryptionkey@0"
I20260811 17:19:21.963126 14592 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260811 17:19:21.965894 14622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:21.967212 14592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:19:21.967386 14592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "f0f10aa1c67c49bdacfa9171323d15cf"
format_stamp: "Formatted at 2026-08-11 17:19:21 on dist-test-slave-mjjr"
server_key: "8594db94d20ffdf1dd53c0474ab84174"
server_key_iv: "04ca54cc5b25e97fee7d745691b8aea0"
server_key_version: "encryptionkey@0"
I20260811 17:19:21.967527 14592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:21.992740 14592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:21.996325 14592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:21.996506 14592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:21.997087 14592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:21.997994 14592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:21.998072 14592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:21.998147 14592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:21.998190 14592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:22.008049 14592 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:35485
I20260811 17:19:22.008074 14813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:35485 every 8 connection(s)
I20260811 17:19:22.009042 14592 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:19:22.011605 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 14592
I20260811 17:19:22.011754 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance
I20260811 17:19:22.012120 9846 external_mini_cluster.cc:1428] Setting key afbef1bef825d7dbf779ea6d60926b5e
Aug 11 17:19:22 dist-test-slave-mjjr krb5kdc[13903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468761, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:22.024101 14064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:22.011158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:53529 (local address 127.9.157.190:46055)
0811 17:19:22.011357 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:22.011361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:22.012438 (+ 1077us) server_negotiation.cc:407] Connection header received
0811 17:19:22.014213 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:22.014217 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:22.014274 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:22.014382 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:22.015790 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:22.016355 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:22.017524 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:22.017745 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:22.017839 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:22.020562 (+ 2723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:22.020566 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:22.020568 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:22.020596 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:22.022012 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:22.022533 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:22.022536 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:22.022537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:22.022617 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:22.022970 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:22.022973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:22.022975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:22.023183 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:22.023276 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:22.023846 (+ 570us) server_negotiation.cc:299] Negotiation successful
0811 17:19:22.023964 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 17:19:22.025134 14819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:22.011671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:46055 (local address 127.9.157.131:53529)
0811 17:19:22.012278 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:22.012327 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:22.013966 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:22.014587 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:22.014595 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:22.015003 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:22.015542 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:22.015557 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:22.016493 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:22.016496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:22.017397 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:22.017407 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:22.017925 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:22.017929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:22.018080 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:22.018678 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:22.018695 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:22.020364 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:22.022124 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:22.022130 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:22.022144 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:22.022414 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:22.022703 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:22.022705 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:22.022707 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:22.022842 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:22.023283 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:22.023288 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:22.023544 (+ 256us) client_negotiation.cc:769] Sending connection context
0811 17:19:22.023772 (+ 228us) client_negotiation.cc:240] Negotiation successful
0811 17:19:22.024074 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":159,"threads_started":1}
I20260811 17:19:22.026360 14815 heartbeater.cc:347] Connected to a master server at 127.9.157.190:46055
I20260811 17:19:22.026616 14815 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:22.027098 14815 heartbeater.cc:511] Master 127.9.157.190:46055 requested a full tablet report, sending...
I20260811 17:19:22.028146 13983 ts_manager.cc:194] Registered new tserver with Master: f0f10aa1c67c49bdacfa9171323d15cf (127.9.157.131:35485)
I20260811 17:19:22.028713 13983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.131@KRBTEST.COM'} at 127.9.157.131:53529
I20260811 17:19:22.038622 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 17:19:22.055732 14064 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:22.053843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38784 (local address 127.9.157.190:46055)
0811 17:19:22.054074 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:22.054081 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:22.054254 (+ 173us) server_negotiation.cc:407] Connection header received
0811 17:19:22.054461 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:22.054466 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:22.054515 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:22.054621 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:22.055528 (+ 907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38784: BlockingRecv error: recv got EOF from 127.0.0.1:38784 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
I20260811 17:19:22.057452 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 14103
I20260811 17:19:22.081676 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 14334
I20260811 17:19:22.094558 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 14592
I20260811 17:19:22.109334 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 13930
2026-08-11T17:19:22Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2563 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:19:22 dist-test-slave-mjjr krb5kdc[14835](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:19:22 dist-test-slave-mjjr krb5kdc[14835](info): set up 2 sockets
Aug 11 17:19:22 dist-test-slave-mjjr krb5kdc[14835](info): commencing operation
krb5kdc: starting...
W20260811 17:19:23.414485 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.274s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:19:23 dist-test-slave-mjjr krb5kdc[14835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:19:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:19:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:19:23.555209 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:37017
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:43277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:37017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:23.677014 14859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:23.677390 14859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:23.677475 14859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:23.681743 14859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:19:23.681872 14859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:23.681913 14859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:23.681952 14859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:19:23.681988 14859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:19:23.687633 14859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:43277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:37017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:37017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:23.688931 14859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:23.690079 14859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:19:23.699597 14859 server_base.cc:1034] running on GCE node
W20260811 17:19:23.699608 14877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:23.699537 14873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:23.699537 14870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:23.700495 14859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:23.701700 14859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:23.702939 14859 hybrid_clock.cc:648] HybridClock initialized: now 1786468763702904 us; error 46 us; skew 500 ppm
Aug 11 17:19:23 dist-test-slave-mjjr krb5kdc[14835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468763, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:23.706323 14859 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:19:23.707474 14859 webserver.cc:460] Webserver started at http://127.9.157.190:35797/ using document root <none> and password file <none>
I20260811 17:19:23.708216 14859 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:23.708328 14859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:23.708612 14859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:23.710988 14859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "d2cb831d259e415db3eae98c751fa59e"
format_stamp: "Formatted at 2026-08-11 17:19:23 on dist-test-slave-mjjr"
server_key: "5cab95c7c021126d23dccb8ad80090cf"
server_key_iv: "2703e90b635fad7d84400f156187c98c"
server_key_version: "encryptionkey@0"
I20260811 17:19:23.711750 14859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "d2cb831d259e415db3eae98c751fa59e"
format_stamp: "Formatted at 2026-08-11 17:19:23 on dist-test-slave-mjjr"
server_key: "5cab95c7c021126d23dccb8ad80090cf"
server_key_iv: "2703e90b635fad7d84400f156187c98c"
server_key_version: "encryptionkey@0"
I20260811 17:19:23.716168 14859 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 17:19:23.718994 14891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:23.720489 14859 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:23.720665 14859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "d2cb831d259e415db3eae98c751fa59e"
format_stamp: "Formatted at 2026-08-11 17:19:23 on dist-test-slave-mjjr"
server_key: "5cab95c7c021126d23dccb8ad80090cf"
server_key_iv: "2703e90b635fad7d84400f156187c98c"
server_key_version: "encryptionkey@0"
I20260811 17:19:23.720813 14859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:23.740931 14859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:23.744966 14859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:23.745242 14859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:23.753391 14859 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:37017
I20260811 17:19:23.753789 14984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:37017 every 8 connection(s)
I20260811 17:19:23.754659 14859 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:19:23.757774 14987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:23.764108 14987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2cb831d259e415db3eae98c751fa59e: Bootstrap starting.
I20260811 17:19:23.764171 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 14859
I20260811 17:19:23.764343 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:19:23.764654 9846 external_mini_cluster.cc:1428] Setting key 7681bfedea0b384709f6e1a0f22abae5
I20260811 17:19:23.767225 14987 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2cb831d259e415db3eae98c751fa59e: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:23.768267 14987 log.cc:824] T 00000000000000000000000000000000 P d2cb831d259e415db3eae98c751fa59e: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:23.770740 14987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2cb831d259e415db3eae98c751fa59e: No bootstrap required, opened a new log
Aug 11 17:19:23 dist-test-slave-mjjr krb5kdc[14835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:23.775179 14987 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2cb831d259e415db3eae98c751fa59e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2cb831d259e415db3eae98c751fa59e" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37017 } }
I20260811 17:19:23.775509 14987 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2cb831d259e415db3eae98c751fa59e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:23.775599 14987 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2cb831d259e415db3eae98c751fa59e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2cb831d259e415db3eae98c751fa59e, State: Initialized, Role: FOLLOWER
I20260811 17:19:23.776226 14987 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2cb831d259e415db3eae98c751fa59e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2cb831d259e415db3eae98c751fa59e" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37017 } }
I20260811 17:19:23.776419 14987 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2cb831d259e415db3eae98c751fa59e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:19:23.776502 14987 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2cb831d259e415db3eae98c751fa59e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:19:23.776609 14987 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2cb831d259e415db3eae98c751fa59e [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:23.778007 14987 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2cb831d259e415db3eae98c751fa59e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2cb831d259e415db3eae98c751fa59e" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37017 } }
I20260811 17:19:23.778417 14987 leader_election.cc:302] T 00000000000000000000000000000000 P d2cb831d259e415db3eae98c751fa59e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2cb831d259e415db3eae98c751fa59e; no voters:
I20260811 17:19:23.778825 14987 leader_election.cc:288] T 00000000000000000000000000000000 P d2cb831d259e415db3eae98c751fa59e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:19:23.778957 14996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2cb831d259e415db3eae98c751fa59e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:23.779258 14996 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2cb831d259e415db3eae98c751fa59e [term 1 LEADER]: Becoming Leader. State: Replica: d2cb831d259e415db3eae98c751fa59e, State: Running, Role: LEADER
I20260811 17:19:23.779685 14996 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2cb831d259e415db3eae98c751fa59e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2cb831d259e415db3eae98c751fa59e" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37017 } }
I20260811 17:19:23.780380 14987 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2cb831d259e415db3eae98c751fa59e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:19:23.781764 14997 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2cb831d259e415db3eae98c751fa59e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2cb831d259e415db3eae98c751fa59e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2cb831d259e415db3eae98c751fa59e" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37017 } } }
I20260811 17:19:23.781917 14997 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2cb831d259e415db3eae98c751fa59e [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:23.781991 14992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:23.766268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33384 (local address 127.9.157.190:37017)
0811 17:19:23.766818 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:23.766839 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:23.766887 (+ 48us) server_negotiation.cc:407] Connection header received
0811 17:19:23.767891 (+ 1004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:23.767919 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:23.768395 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:23.768803 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:23.769782 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:23.771217 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:23.772202 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:23.772951 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:23.773123 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:23.775760 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:23.775767 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:23.775786 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:23.775824 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:23.778162 (+ 2338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:23.778767 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:23.778775 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:23.778782 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:23.778945 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:23.779315 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:23.779321 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:23.779326 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:23.779951 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:23.780164 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:23.780497 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 17:19:23.780822 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":180,"threads_started":1}
I20260811 17:19:23.782377 15009 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:19:23.782500 14998 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2cb831d259e415db3eae98c751fa59e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2cb831d259e415db3eae98c751fa59e. Latest consensus state: current_term: 1 leader_uuid: "d2cb831d259e415db3eae98c751fa59e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2cb831d259e415db3eae98c751fa59e" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37017 } } }
I20260811 17:19:23.782698 14998 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2cb831d259e415db3eae98c751fa59e [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:23.785578 9846 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:19:23.786761 15009 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:19:23.793406 15009 catalog_manager.cc:1347] Generated new cluster ID: 287973e90b5644ce87323c3f49c8c801
I20260811 17:19:23.793489 15009 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:19:23.807156 15009 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:19:23.808832 15009 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:19:23.818054 15009 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2cb831d259e415db3eae98c751fa59e: Generated new TSK 0
I20260811 17:19:23.818138 14992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:23.810841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33390 (local address 127.9.157.190:37017)
0811 17:19:23.810953 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:23.810958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:23.811244 (+ 286us) server_negotiation.cc:407] Connection header received
0811 17:19:23.811386 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:23.811389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:23.811446 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:23.811540 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:23.812201 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:23.813042 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:23.814399 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:23.814783 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:23.814898 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:23.816002 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:23.816006 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:23.816009 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:19:23.816791 (+ 782us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:23.816843 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:19:23.816868 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:23.816990 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:23.817166 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 17:19:23.817731 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 17:19:23.819262 15009 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:19:23.831529 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 14859
2026-08-11T17:19:23Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1720 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:19:23 dist-test-slave-mjjr krb5kdc[15044](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:19:23 dist-test-slave-mjjr krb5kdc[15044](info): set up 2 sockets
Aug 11 17:19:23 dist-test-slave-mjjr krb5kdc[15044](info): commencing operation
krb5kdc: starting...
W20260811 17:19:25.129405 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:19:25 dist-test-slave-mjjr krb5kdc[15044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:19:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:19:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:19:25.270841 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:46795
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:42931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:46795
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:25.396713 15065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:25.397028 15065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:25.397070 15065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:25.401494 15065 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:19:25.401573 15065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:25.401593 15065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:25.401614 15065 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:19:25.401630 15065 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:19:25.407305 15065 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:42931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:46795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:46795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:25.408615 15065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:25.409756 15065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:25.419044 15080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:25.419067 15078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:25.419067 15076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:25.419454 15065 server_base.cc:1034] running on GCE node
I20260811 17:19:25.420173 15065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:25.421430 15065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:25.422636 15065 hybrid_clock.cc:648] HybridClock initialized: now 1786468765422599 us; error 51 us; skew 500 ppm
Aug 11 17:19:25 dist-test-slave-mjjr krb5kdc[15044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468765, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:25.429961 15065 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:19:25.431169 15065 webserver.cc:460] Webserver started at http://127.9.157.190:34659/ using document root <none> and password file <none>
I20260811 17:19:25.431936 15065 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:25.432045 15065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:25.432332 15065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:25.435264 15065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "1d88dd7521314e3a9ef24bea2fb978e1"
format_stamp: "Formatted at 2026-08-11 17:19:25 on dist-test-slave-mjjr"
server_key: "e06ed35015b5a7ee8183d44e2f5c38cc"
server_key_iv: "6e4150523b0d91128342244b5ff26dc0"
server_key_version: "encryptionkey@0"
I20260811 17:19:25.436028 15065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "1d88dd7521314e3a9ef24bea2fb978e1"
format_stamp: "Formatted at 2026-08-11 17:19:25 on dist-test-slave-mjjr"
server_key: "e06ed35015b5a7ee8183d44e2f5c38cc"
server_key_iv: "6e4150523b0d91128342244b5ff26dc0"
server_key_version: "encryptionkey@0"
I20260811 17:19:25.440768 15065 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 17:19:25.443612 15092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:25.445108 15065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 17:19:25.445288 15065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "1d88dd7521314e3a9ef24bea2fb978e1"
format_stamp: "Formatted at 2026-08-11 17:19:25 on dist-test-slave-mjjr"
server_key: "e06ed35015b5a7ee8183d44e2f5c38cc"
server_key_iv: "6e4150523b0d91128342244b5ff26dc0"
server_key_version: "encryptionkey@0"
I20260811 17:19:25.445422 15065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:25.471367 15065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:25.474852 15065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:25.475097 15065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:25.483325 15065 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:46795
I20260811 17:19:25.483672 15192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:46795 every 8 connection(s)
I20260811 17:19:25.484580 15065 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:19:25.487881 15194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:25.490667 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 15065
I20260811 17:19:25.490796 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:19:25.491148 9846 external_mini_cluster.cc:1428] Setting key ca44f97a3f9f8dc4aba9fe64057612e6
I20260811 17:19:25.494583 15194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d88dd7521314e3a9ef24bea2fb978e1: Bootstrap starting.
I20260811 17:19:25.497587 15194 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d88dd7521314e3a9ef24bea2fb978e1: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:25.498656 15194 log.cc:824] T 00000000000000000000000000000000 P 1d88dd7521314e3a9ef24bea2fb978e1: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:25.501014 15194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d88dd7521314e3a9ef24bea2fb978e1: No bootstrap required, opened a new log
Aug 11 17:19:25 dist-test-slave-mjjr krb5kdc[15044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:25.505983 15194 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d88dd7521314e3a9ef24bea2fb978e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d88dd7521314e3a9ef24bea2fb978e1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 46795 } }
I20260811 17:19:25.506336 15194 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d88dd7521314e3a9ef24bea2fb978e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:25.506405 15194 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d88dd7521314e3a9ef24bea2fb978e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d88dd7521314e3a9ef24bea2fb978e1, State: Initialized, Role: FOLLOWER
I20260811 17:19:25.507103 15194 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d88dd7521314e3a9ef24bea2fb978e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d88dd7521314e3a9ef24bea2fb978e1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 46795 } }
I20260811 17:19:25.507349 15194 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d88dd7521314e3a9ef24bea2fb978e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:19:25.507436 15194 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d88dd7521314e3a9ef24bea2fb978e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:19:25.507586 15194 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d88dd7521314e3a9ef24bea2fb978e1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:25.509186 15194 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d88dd7521314e3a9ef24bea2fb978e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d88dd7521314e3a9ef24bea2fb978e1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 46795 } }
I20260811 17:19:25.509618 15194 leader_election.cc:302] T 00000000000000000000000000000000 P 1d88dd7521314e3a9ef24bea2fb978e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d88dd7521314e3a9ef24bea2fb978e1; no voters:
I20260811 17:19:25.510131 15194 leader_election.cc:288] T 00000000000000000000000000000000 P 1d88dd7521314e3a9ef24bea2fb978e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:19:25.510588 15205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d88dd7521314e3a9ef24bea2fb978e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:25.510883 15205 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d88dd7521314e3a9ef24bea2fb978e1 [term 1 LEADER]: Becoming Leader. State: Replica: 1d88dd7521314e3a9ef24bea2fb978e1, State: Running, Role: LEADER
I20260811 17:19:25.511373 15205 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d88dd7521314e3a9ef24bea2fb978e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d88dd7521314e3a9ef24bea2fb978e1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 46795 } }
I20260811 17:19:25.511485 15201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:25.493019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33210 (local address 127.9.157.190:46795)
0811 17:19:25.493529 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:25.493546 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:25.493584 (+ 38us) server_negotiation.cc:407] Connection header received
0811 17:19:25.494236 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:25.494254 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:25.494607 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:25.494988 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:25.496006 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:25.497129 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:25.498410 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:25.499011 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:25.499194 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:25.501864 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:25.501871 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:25.501892 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:25.501929 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:25.505170 (+ 3241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:25.506649 (+ 1479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:25.506657 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:25.506663 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:25.506823 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:25.507285 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:25.507290 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:25.507292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:25.509242 (+ 1950us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:25.509517 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:25.509978 (+ 461us) server_negotiation.cc:299] Negotiation successful
0811 17:19:25.510400 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":183,"threads_started":1}
I20260811 17:19:25.511632 15194 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d88dd7521314e3a9ef24bea2fb978e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:19:25.515892 15208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d88dd7521314e3a9ef24bea2fb978e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d88dd7521314e3a9ef24bea2fb978e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d88dd7521314e3a9ef24bea2fb978e1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 46795 } } }
I20260811 17:19:25.516192 15208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d88dd7521314e3a9ef24bea2fb978e1 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:25.516273 15204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d88dd7521314e3a9ef24bea2fb978e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d88dd7521314e3a9ef24bea2fb978e1. Latest consensus state: current_term: 1 leader_uuid: "1d88dd7521314e3a9ef24bea2fb978e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d88dd7521314e3a9ef24bea2fb978e1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 46795 } } }
I20260811 17:19:25.516425 15204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d88dd7521314e3a9ef24bea2fb978e1 [sys.catalog]: This master's current role is: LEADER
W20260811 17:19:25.521485 15230 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1d88dd7521314e3a9ef24bea2fb978e1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:19:25.521606 15230 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:19:25.521776 15231 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:19:25.522823 15231 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:19:25.530006 15231 catalog_manager.cc:1347] Generated new cluster ID: 4c4192a9a1ae4b50bb5f1a507146642e
I20260811 17:19:25.530113 15231 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:19:25.557178 15231 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:19:25.558265 15231 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:19:25.568996 15231 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d88dd7521314e3a9ef24bea2fb978e1: Generated new TSK 0
I20260811 17:19:25.569833 15231 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
I20260811 17:19:25.612149 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:0
--local_ip_for_outbound_sockets=127.9.157.129
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:46795
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:42931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:25.753974 15240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:25.754295 15240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:25.754343 15240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:25.758495 15240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:25.758567 15240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:25.758626 15240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:19:25.764083 15240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:42931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:46795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:25.765229 15240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:25.766263 15240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:25.778647 15249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:25.778681 15240 server_base.cc:1034] running on GCE node
W20260811 17:19:25.778671 15253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:25.779142 15248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:25.779691 15240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:25.780400 15240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:25.781582 15240 hybrid_clock.cc:648] HybridClock initialized: now 1786468765781564 us; error 27 us; skew 500 ppm
Aug 11 17:19:25 dist-test-slave-mjjr krb5kdc[15044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468765, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:25.784760 15240 init.cc:377] Logged in from keytab as kudu/127.9.157.129@KRBTEST.COM (short username kudu)
I20260811 17:19:25.785801 15240 webserver.cc:460] Webserver started at http://127.9.157.129:33585/ using document root <none> and password file <none>
I20260811 17:19:25.786470 15240 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:25.786574 15240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:25.786849 15240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:25.789163 15240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/data/instance:
uuid: "c4203dccd5ef44108c08d23dcc894ba4"
format_stamp: "Formatted at 2026-08-11 17:19:25 on dist-test-slave-mjjr"
server_key: "bb3b9ba1432d2edd664f09f35007f9a4"
server_key_iv: "39106d7a63a855022181dc40288c02b4"
server_key_version: "encryptionkey@0"
I20260811 17:19:25.789781 15240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance:
uuid: "c4203dccd5ef44108c08d23dcc894ba4"
format_stamp: "Formatted at 2026-08-11 17:19:25 on dist-test-slave-mjjr"
server_key: "bb3b9ba1432d2edd664f09f35007f9a4"
server_key_iv: "39106d7a63a855022181dc40288c02b4"
server_key_version: "encryptionkey@0"
I20260811 17:19:25.793931 15240 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:19:25.796653 15264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:25.798012 15240 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:25.798298 15240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "c4203dccd5ef44108c08d23dcc894ba4"
format_stamp: "Formatted at 2026-08-11 17:19:25 on dist-test-slave-mjjr"
server_key: "bb3b9ba1432d2edd664f09f35007f9a4"
server_key_iv: "39106d7a63a855022181dc40288c02b4"
server_key_version: "encryptionkey@0"
I20260811 17:19:25.798461 15240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:25.838085 15240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:25.841384 15240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:25.841622 15240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:25.842321 15240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:25.843245 15240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:25.843320 15240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:25.843423 15240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:25.843475 15240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:25.853307 15240 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:46267
I20260811 17:19:25.853329 15491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:46267 every 8 connection(s)
I20260811 17:19:25.854278 15240 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:19:25.862913 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 15240
I20260811 17:19:25.863065 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance
I20260811 17:19:25.863425 9846 external_mini_cluster.cc:1428] Setting key 9111b18b690704f74c6523d97a2dd38e
Aug 11 17:19:25 dist-test-slave-mjjr krb5kdc[15044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468765, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:25.872215 15201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:25.856283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:37849 (local address 127.9.157.190:46795)
0811 17:19:25.856443 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:25.856447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:25.857362 (+ 915us) server_negotiation.cc:407] Connection header received
0811 17:19:25.858934 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:25.858938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:25.859002 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:25.859109 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:25.860488 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:25.861049 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:25.862270 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:25.862524 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:25.862655 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:25.866201 (+ 3546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:25.866205 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:25.866209 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:25.866246 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:25.868238 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:25.869003 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:25.869010 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:25.869014 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:25.869117 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:25.869614 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:25.869618 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:25.869620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:25.869967 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:25.870086 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:25.871825 (+ 1739us) server_negotiation.cc:299] Negotiation successful
0811 17:19:25.872068 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 17:19:25.872576 15496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:25.856702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:46795 (local address 127.9.157.129:37849)
0811 17:19:25.857177 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:25.857223 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:25.858638 (+ 1415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:25.859303 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:25.859311 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:25.859787 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:25.860308 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:25.860322 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:25.861195 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:25.861199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:25.862131 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:25.862142 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:25.862886 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:25.862891 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:25.863119 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:25.863893 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:25.863918 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:25.865989 (+ 2071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:25.868403 (+ 2414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:25.868412 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:25.868446 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:25.868865 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:25.869229 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:25.869232 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:25.869234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:25.869468 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:25.870111 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:25.870121 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:25.870520 (+ 399us) client_negotiation.cc:769] Sending connection context
0811 17:19:25.870765 (+ 245us) client_negotiation.cc:240] Negotiation successful
0811 17:19:25.871207 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":129,"threads_started":1}
I20260811 17:19:25.874028 15493 heartbeater.cc:347] Connected to a master server at 127.9.157.190:46795
I20260811 17:19:25.874300 15493 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:25.874859 15493 heartbeater.cc:511] Master 127.9.157.190:46795 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.157.130@KRBTEST.COM; defaulting to no policy
I20260811 17:19:25.876559 15119 ts_manager.cc:194] Registered new tserver with Master: c4203dccd5ef44108c08d23dcc894ba4 (127.9.157.129:46267)
I20260811 17:19:25.877828 15119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.129@KRBTEST.COM'} at 127.9.157.129:37849
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
I20260811 17:19:25.897022 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:0
--local_ip_for_outbound_sockets=127.9.157.130
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:46795
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:42931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:26.020699 15500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:26.021070 15500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:26.021181 15500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:26.025369 15500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:26.025466 15500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:26.025556 15500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:19:26.031210 15500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:42931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:46795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:26.032581 15500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:26.033730 15500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:26.044080 15505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:26.044085 15506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:26.044080 15510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:26.044117 15500 server_base.cc:1034] running on GCE node
I20260811 17:19:26.044816 15500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:26.045640 15500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:26.046839 15500 hybrid_clock.cc:648] HybridClock initialized: now 1786468766046801 us; error 59 us; skew 500 ppm
Aug 11 17:19:26 dist-test-slave-mjjr krb5kdc[15044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468766, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:26.049870 15500 init.cc:377] Logged in from keytab as kudu/127.9.157.130@KRBTEST.COM (short username kudu)
I20260811 17:19:26.050915 15500 webserver.cc:460] Webserver started at http://127.9.157.130:34289/ using document root <none> and password file <none>
I20260811 17:19:26.051541 15500 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:26.051611 15500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:26.051890 15500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:26.054102 15500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/data/instance:
uuid: "f2b8037a58c349e692bd50115585fe66"
format_stamp: "Formatted at 2026-08-11 17:19:26 on dist-test-slave-mjjr"
server_key: "951abfcd9e32575d38b51ac943b467c2"
server_key_iv: "0dcbda59ae1d3e69fc443101652e0fdf"
server_key_version: "encryptionkey@0"
I20260811 17:19:26.054735 15500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance:
uuid: "f2b8037a58c349e692bd50115585fe66"
format_stamp: "Formatted at 2026-08-11 17:19:26 on dist-test-slave-mjjr"
server_key: "951abfcd9e32575d38b51ac943b467c2"
server_key_iv: "0dcbda59ae1d3e69fc443101652e0fdf"
server_key_version: "encryptionkey@0"
I20260811 17:19:26.058858 15500 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260811 17:19:26.061584 15520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:26.062984 15500 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:19:26.063233 15500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "f2b8037a58c349e692bd50115585fe66"
format_stamp: "Formatted at 2026-08-11 17:19:26 on dist-test-slave-mjjr"
server_key: "951abfcd9e32575d38b51ac943b467c2"
server_key_iv: "0dcbda59ae1d3e69fc443101652e0fdf"
server_key_version: "encryptionkey@0"
I20260811 17:19:26.063364 15500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:26.095098 15500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:26.098484 15500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:26.098718 15500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:26.099466 15500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:26.100433 15500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:26.100508 15500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:26.100579 15500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:26.100626 15500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:26.110292 15500 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:37135
I20260811 17:19:26.110312 15700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:37135 every 8 connection(s)
I20260811 17:19:26.111285 15500 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:19:26.115697 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 15500
I20260811 17:19:26.115834 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance
I20260811 17:19:26.116153 9846 external_mini_cluster.cc:1428] Setting key bf3095e7b4187d77129f30e3699e4de8
Aug 11 17:19:26 dist-test-slave-mjjr krb5kdc[15044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468766, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.157.131@KRBTEST.COM; defaulting to no policy
I20260811 17:19:26.129101 15201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:26.113390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:36905 (local address 127.9.157.190:46795)
0811 17:19:26.113540 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:26.113544 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:26.114485 (+ 941us) server_negotiation.cc:407] Connection header received
0811 17:19:26.116387 (+ 1902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:26.116391 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:26.116447 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:26.116529 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:26.118117 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:26.118642 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:26.120485 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:26.120922 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:26.121057 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:26.124381 (+ 3324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:26.124388 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:26.124395 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:26.124439 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:26.126278 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:26.127025 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:26.127031 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:26.127035 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:26.127142 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:26.127589 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:26.127595 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:26.127600 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:26.127914 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:26.128041 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:26.128705 (+ 664us) server_negotiation.cc:299] Negotiation successful
0811 17:19:26.128877 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 17:19:26.130236 15704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:26.113786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:46795 (local address 127.9.157.130:36905)
0811 17:19:26.114312 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:26.114361 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:26.116071 (+ 1710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:26.116750 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:26.116758 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:26.117248 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:26.117956 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:26.117979 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:26.118788 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:26.118794 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:26.120345 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:26.120356 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:26.121104 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:26.121107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:26.121261 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:26.121957 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:26.121974 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:26.124117 (+ 2143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:26.126466 (+ 2349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:26.126472 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:26.126490 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:26.126842 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:26.127289 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:26.127292 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:26.127293 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:26.127456 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:26.128068 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:26.128075 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:26.128393 (+ 318us) client_negotiation.cc:769] Sending connection context
0811 17:19:26.128582 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 17:19:26.129003 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":119,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.131@KRBTEST.COM" created.
I20260811 17:19:26.131843 15702 heartbeater.cc:347] Connected to a master server at 127.9.157.190:46795
I20260811 17:19:26.132159 15702 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:26.132874 15702 heartbeater.cc:511] Master 127.9.157.190:46795 requested a full tablet report, sending...
I20260811 17:19:26.134479 15119 ts_manager.cc:194] Registered new tserver with Master: f2b8037a58c349e692bd50115585fe66 (127.9.157.130:37135)
I20260811 17:19:26.135504 15119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.130@KRBTEST.COM'} at 127.9.157.130:36905
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
I20260811 17:19:26.150408 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:0
--local_ip_for_outbound_sockets=127.9.157.131
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:46795
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:42931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:26.275257 15710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:26.275628 15710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:26.275756 15710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:26.279942 15710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:26.280064 15710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:26.280174 15710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:19:26.285734 15710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:42931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:46795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:26.287021 15710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:26.288228 15710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:26.299088 15715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:26.299137 15719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:26.299137 15716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:26.299458 15710 server_base.cc:1034] running on GCE node
I20260811 17:19:26.300047 15710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:26.300805 15710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:26.302012 15710 hybrid_clock.cc:648] HybridClock initialized: now 1786468766301989 us; error 33 us; skew 500 ppm
Aug 11 17:19:26 dist-test-slave-mjjr krb5kdc[15044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468766, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:26.305303 15710 init.cc:377] Logged in from keytab as kudu/127.9.157.131@KRBTEST.COM (short username kudu)
I20260811 17:19:26.306320 15710 webserver.cc:460] Webserver started at http://127.9.157.131:37153/ using document root <none> and password file <none>
I20260811 17:19:26.306979 15710 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:26.307085 15710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:26.307348 15710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:26.309722 15710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/data/instance:
uuid: "6663e20917a7428db939f1880aa27324"
format_stamp: "Formatted at 2026-08-11 17:19:26 on dist-test-slave-mjjr"
server_key: "b3d0e684d74a69e473e2a1b1569ea01c"
server_key_iv: "8b08f38c6b85a831728b6088542e959a"
server_key_version: "encryptionkey@0"
I20260811 17:19:26.310470 15710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance:
uuid: "6663e20917a7428db939f1880aa27324"
format_stamp: "Formatted at 2026-08-11 17:19:26 on dist-test-slave-mjjr"
server_key: "b3d0e684d74a69e473e2a1b1569ea01c"
server_key_iv: "8b08f38c6b85a831728b6088542e959a"
server_key_version: "encryptionkey@0"
I20260811 17:19:26.315053 15710 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:19:26.317919 15734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:26.319218 15710 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:19:26.319392 15710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "6663e20917a7428db939f1880aa27324"
format_stamp: "Formatted at 2026-08-11 17:19:26 on dist-test-slave-mjjr"
server_key: "b3d0e684d74a69e473e2a1b1569ea01c"
server_key_iv: "8b08f38c6b85a831728b6088542e959a"
server_key_version: "encryptionkey@0"
I20260811 17:19:26.319536 15710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:26.333006 15710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:26.336313 15710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:26.336556 15710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:26.337234 15710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:26.338166 15710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:26.338246 15710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:26.338320 15710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:26.338373 15710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:26.348553 15710 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:38869
I20260811 17:19:26.348583 15921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:38869 every 8 connection(s)
I20260811 17:19:26.349700 15710 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:19:26.359694 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 15710
I20260811 17:19:26.359875 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance
I20260811 17:19:26.360257 9846 external_mini_cluster.cc:1428] Setting key 99faccaefd6043ce59c88b9b7cb48a36
Aug 11 17:19:26 dist-test-slave-mjjr krb5kdc[15044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468766, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:26.365945 15201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:26.352118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:43279 (local address 127.9.157.190:46795)
0811 17:19:26.352341 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:26.352347 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:26.353396 (+ 1049us) server_negotiation.cc:407] Connection header received
0811 17:19:26.354975 (+ 1579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:26.354978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:26.355037 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:26.355131 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:26.356742 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:26.357318 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:26.358574 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:26.358825 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:26.358946 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:26.362140 (+ 3194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:26.362144 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:26.362147 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:26.362181 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:26.363777 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:26.364393 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:26.364395 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:26.364396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:26.364486 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:26.364869 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:26.364871 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:26.364872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:26.365068 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:26.365147 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:26.365686 (+ 539us) server_negotiation.cc:299] Negotiation successful
0811 17:19:26.365814 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 17:19:26.366914 15930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:26.352591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:46795 (local address 127.9.157.131:43279)
0811 17:19:26.353190 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:26.353252 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:26.354694 (+ 1442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:26.355436 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:26.355447 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:26.355993 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:26.356548 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:26.356564 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:26.357465 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:26.357468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:26.358426 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:26.358438 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:26.359000 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:26.359003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:26.359156 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:26.360131 (+ 975us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:26.360156 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:26.361970 (+ 1814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:26.363926 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:26.363932 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:26.363948 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:26.364288 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:26.364598 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:26.364601 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:26.364603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:26.364776 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:26.365163 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:26.365169 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:26.365449 (+ 280us) client_negotiation.cc:769] Sending connection context
0811 17:19:26.365625 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 17:19:26.365945 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":166,"threads_started":1}
I20260811 17:19:26.368367 15925 heartbeater.cc:347] Connected to a master server at 127.9.157.190:46795
I20260811 17:19:26.368669 15925 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:26.369271 15925 heartbeater.cc:511] Master 127.9.157.190:46795 requested a full tablet report, sending...
I20260811 17:19:26.370428 15119 ts_manager.cc:194] Registered new tserver with Master: 6663e20917a7428db939f1880aa27324 (127.9.157.131:38869)
I20260811 17:19:26.371054 15119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.131@KRBTEST.COM'} at 127.9.157.131:43279
I20260811 17:19:26.374655 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 17:19:26 dist-test-slave-mjjr krb5kdc[15044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468766, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 17:19:26 dist-test-slave-mjjr krb5kdc[15044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468766, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:26.403910 15201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:26.394722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33214 (local address 127.9.157.190:46795)
0811 17:19:26.394928 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:26.394933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:26.395087 (+ 154us) server_negotiation.cc:407] Connection header received
0811 17:19:26.395281 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:26.395284 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:26.395333 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:26.395429 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:26.396282 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:26.396827 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:26.397689 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:26.397914 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:26.398030 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:26.400702 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:26.400706 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:26.400708 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:26.400737 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:26.402060 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:26.402580 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:26.402583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:26.402585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:26.402664 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:26.402903 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:26.402905 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:26.402907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:26.403204 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:26.403371 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:26.403629 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 17:19:26.403781 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260811 17:19:26.404368 15119 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:33214
I20260811 17:19:26.405306 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 15240
I20260811 17:19:26.420037 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 15500
I20260811 17:19:26.444283 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 15710
I20260811 17:19:26.464484 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 15065
2026-08-11T17:19:26Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2633 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:19:26 dist-test-slave-mjjr krb5kdc[15947](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:19:26 dist-test-slave-mjjr krb5kdc[15947](info): set up 2 sockets
Aug 11 17:19:26 dist-test-slave-mjjr krb5kdc[15947](info): commencing operation
krb5kdc: starting...
W20260811 17:19:27.738461 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:19:27 dist-test-slave-mjjr krb5kdc[15947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:19:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:19:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:19:27.885119 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:36801
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:38065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:36801
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:28.009321 15968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:28.009618 15968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:28.009661 15968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:28.013931 15968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:19:28.014006 15968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:28.014026 15968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:28.014045 15968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:19:28.014061 15968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:19:28.019809 15968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:38065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:36801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:36801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:28.021389 15968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:28.022626 15968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:28.031801 15976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:28.031801 15974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:28.031821 15980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:28.032125 15968 server_base.cc:1034] running on GCE node
I20260811 17:19:28.032881 15968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:28.034078 15968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:28.035271 15968 hybrid_clock.cc:648] HybridClock initialized: now 1786468768035246 us; error 43 us; skew 500 ppm
Aug 11 17:19:28 dist-test-slave-mjjr krb5kdc[15947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468768, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:28.038399 15968 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:19:28.039489 15968 webserver.cc:460] Webserver started at http://127.9.157.190:37877/ using document root <none> and password file <none>
I20260811 17:19:28.040197 15968 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:28.040298 15968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:28.040568 15968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:28.042821 15968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "7332796b404945c692e434ab9f26887a"
format_stamp: "Formatted at 2026-08-11 17:19:28 on dist-test-slave-mjjr"
server_key: "26515e5288e7bba110bf0cbed6df6756"
server_key_iv: "a3303640d09513cff8d77109c8d79cff"
server_key_version: "encryptionkey@0"
I20260811 17:19:28.043464 15968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "7332796b404945c692e434ab9f26887a"
format_stamp: "Formatted at 2026-08-11 17:19:28 on dist-test-slave-mjjr"
server_key: "26515e5288e7bba110bf0cbed6df6756"
server_key_iv: "a3303640d09513cff8d77109c8d79cff"
server_key_version: "encryptionkey@0"
I20260811 17:19:28.047621 15968 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 17:19:28.050415 15992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:28.051856 15968 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 17:19:28.052012 15968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "7332796b404945c692e434ab9f26887a"
format_stamp: "Formatted at 2026-08-11 17:19:28 on dist-test-slave-mjjr"
server_key: "26515e5288e7bba110bf0cbed6df6756"
server_key_iv: "a3303640d09513cff8d77109c8d79cff"
server_key_version: "encryptionkey@0"
I20260811 17:19:28.052143 15968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:28.069813 15968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:28.073144 15968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:28.073414 15968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:28.081578 15968 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:36801
I20260811 17:19:28.081597 16072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:36801 every 8 connection(s)
I20260811 17:19:28.082652 15968 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:19:28.083822 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 15968
I20260811 17:19:28.083971 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:19:28.084275 9846 external_mini_cluster.cc:1428] Setting key 0c7b7478a2cd918b3a952694fcf54d7c
I20260811 17:19:28.086476 16073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:28.094712 16073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7332796b404945c692e434ab9f26887a: Bootstrap starting.
Aug 11 17:19:28 dist-test-slave-mjjr krb5kdc[15947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:28.097823 16073 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7332796b404945c692e434ab9f26887a: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:28.098855 16073 log.cc:824] T 00000000000000000000000000000000 P 7332796b404945c692e434ab9f26887a: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:28.101578 16073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7332796b404945c692e434ab9f26887a: No bootstrap required, opened a new log
I20260811 17:19:28.101854 16077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:28.085907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34674 (local address 127.9.157.190:36801)
0811 17:19:28.086419 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:28.086460 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:28.086514 (+ 54us) server_negotiation.cc:407] Connection header received
0811 17:19:28.087485 (+ 971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:28.087512 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:28.088027 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:28.088463 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:28.089277 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:28.090987 (+ 1710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:28.092056 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:28.092878 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:28.093070 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:28.096285 (+ 3215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:28.096291 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:28.096308 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:28.096348 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:28.098320 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:28.098940 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:28.098945 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:28.098951 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:28.099116 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:28.099463 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:28.099466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:28.099469 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:28.100137 (+ 668us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:28.100294 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:28.100649 (+ 355us) server_negotiation.cc:299] Negotiation successful
0811 17:19:28.100996 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":343,"thread_start_us":147,"threads_started":1}
I20260811 17:19:28.105688 16073 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7332796b404945c692e434ab9f26887a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7332796b404945c692e434ab9f26887a" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 36801 } }
I20260811 17:19:28.105968 16073 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7332796b404945c692e434ab9f26887a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:28.106058 16073 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7332796b404945c692e434ab9f26887a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7332796b404945c692e434ab9f26887a, State: Initialized, Role: FOLLOWER
I20260811 17:19:28.106546 16073 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7332796b404945c692e434ab9f26887a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7332796b404945c692e434ab9f26887a" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 36801 } }
I20260811 17:19:28.106686 16073 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7332796b404945c692e434ab9f26887a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:19:28.106784 16073 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7332796b404945c692e434ab9f26887a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:19:28.106930 16073 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7332796b404945c692e434ab9f26887a [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:28.108120 16073 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7332796b404945c692e434ab9f26887a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7332796b404945c692e434ab9f26887a" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 36801 } }
I20260811 17:19:28.108526 16073 leader_election.cc:302] T 00000000000000000000000000000000 P 7332796b404945c692e434ab9f26887a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7332796b404945c692e434ab9f26887a; no voters:
I20260811 17:19:28.108902 16073 leader_election.cc:288] T 00000000000000000000000000000000 P 7332796b404945c692e434ab9f26887a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:19:28.109055 16079 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7332796b404945c692e434ab9f26887a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:28.109341 16079 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7332796b404945c692e434ab9f26887a [term 1 LEADER]: Becoming Leader. State: Replica: 7332796b404945c692e434ab9f26887a, State: Running, Role: LEADER
I20260811 17:19:28.109679 16079 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7332796b404945c692e434ab9f26887a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7332796b404945c692e434ab9f26887a" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 36801 } }
I20260811 17:19:28.110394 16073 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7332796b404945c692e434ab9f26887a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:19:28.111527 16085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7332796b404945c692e434ab9f26887a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7332796b404945c692e434ab9f26887a. Latest consensus state: current_term: 1 leader_uuid: "7332796b404945c692e434ab9f26887a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7332796b404945c692e434ab9f26887a" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 36801 } } }
I20260811 17:19:28.111719 16085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7332796b404945c692e434ab9f26887a [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:28.112067 16084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7332796b404945c692e434ab9f26887a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7332796b404945c692e434ab9f26887a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7332796b404945c692e434ab9f26887a" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 36801 } } }
I20260811 17:19:28.112174 16084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7332796b404945c692e434ab9f26887a [sys.catalog]: This master's current role is: LEADER
W20260811 17:19:28.116189 16103 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7332796b404945c692e434ab9f26887a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:19:28.116304 16103 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:19:28.116452 16094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:19:28.117533 16094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:19:28.124361 16094 catalog_manager.cc:1347] Generated new cluster ID: 8c550a3402674ab1847e875bd4f5f333
I20260811 17:19:28.124461 16094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:19:28.152825 16094 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:19:28.153888 16094 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:19:28.159807 16094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7332796b404945c692e434ab9f26887a: Generated new TSK 0
I20260811 17:19:28.160615 16094 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
I20260811 17:19:28.199445 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:0
--local_ip_for_outbound_sockets=127.9.157.129
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:36801
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:38065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:28.331633 16111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:28.332002 16111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:28.332049 16111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:28.336440 16111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:28.336526 16111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:28.336591 16111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:19:28.342317 16111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:38065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:36801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:28.343614 16111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:28.344852 16111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:19:28.356232 16111 server_base.cc:1034] running on GCE node
W20260811 17:19:28.356134 16123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:28.356199 16120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:28.356141 16125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:28.356972 16111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:28.357658 16111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:28.358882 16111 hybrid_clock.cc:648] HybridClock initialized: now 1786468768358844 us; error 49 us; skew 500 ppm
Aug 11 17:19:28 dist-test-slave-mjjr krb5kdc[15947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468768, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:28.362416 16111 init.cc:377] Logged in from keytab as kudu/127.9.157.129@KRBTEST.COM (short username kudu)
I20260811 17:19:28.363523 16111 webserver.cc:460] Webserver started at http://127.9.157.129:42679/ using document root <none> and password file <none>
I20260811 17:19:28.364204 16111 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:28.364283 16111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:28.364475 16111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:28.366732 16111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-0/data/instance:
uuid: "bfeb4cf68dd548feb3e7cb63045bedc9"
format_stamp: "Formatted at 2026-08-11 17:19:28 on dist-test-slave-mjjr"
server_key: "05acd25bb50e1614415223a0b19ab521"
server_key_iv: "327b17a8d31b3b58a108f54b0653df8f"
server_key_version: "encryptionkey@0"
I20260811 17:19:28.367345 16111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance:
uuid: "bfeb4cf68dd548feb3e7cb63045bedc9"
format_stamp: "Formatted at 2026-08-11 17:19:28 on dist-test-slave-mjjr"
server_key: "05acd25bb50e1614415223a0b19ab521"
server_key_iv: "327b17a8d31b3b58a108f54b0653df8f"
server_key_version: "encryptionkey@0"
I20260811 17:19:28.371838 16111 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260811 17:19:28.374709 16134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:28.376180 16111 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:28.376333 16111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "bfeb4cf68dd548feb3e7cb63045bedc9"
format_stamp: "Formatted at 2026-08-11 17:19:28 on dist-test-slave-mjjr"
server_key: "05acd25bb50e1614415223a0b19ab521"
server_key_iv: "327b17a8d31b3b58a108f54b0653df8f"
server_key_version: "encryptionkey@0"
I20260811 17:19:28.376456 16111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:28.406672 16111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:28.410609 16111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:28.410852 16111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:28.411716 16111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:28.412810 16111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:28.412864 16111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:28.412910 16111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:28.412926 16111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:28.422693 16111 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:43555
I20260811 17:19:28.422771 16328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:43555 every 8 connection(s)
I20260811 17:19:28.423864 16111 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:19:28.429598 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 16111
I20260811 17:19:28.429755 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance
I20260811 17:19:28.430095 9846 external_mini_cluster.cc:1428] Setting key 2f86f8719f243c3e6b78098a9bb09f0b
Aug 11 17:19:28 dist-test-slave-mjjr krb5kdc[15947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468768, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:28.441298 16077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:28.426177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:39247 (local address 127.9.157.190:36801)
0811 17:19:28.426352 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:28.426358 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:28.427393 (+ 1035us) server_negotiation.cc:407] Connection header received
0811 17:19:28.429059 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:28.429062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:28.429116 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:28.429198 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:28.430969 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:28.431771 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:28.433565 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:28.433816 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:28.433960 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:28.437031 (+ 3071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:28.437034 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:28.437036 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:28.437065 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:28.438674 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:28.439450 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:28.439453 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:28.439454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:28.439536 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:28.440013 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:28.440016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:28.440017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:28.440216 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:28.440296 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:28.441044 (+ 748us) server_negotiation.cc:299] Negotiation successful
0811 17:19:28.441166 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
WARNING: no policy specified for kudu/127.9.157.130@KRBTEST.COM; defaulting to no policy
I20260811 17:19:28.442667 16333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:28.426630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:36801 (local address 127.9.157.129:39247)
0811 17:19:28.427213 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:28.427263 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:28.428808 (+ 1545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:28.429456 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:28.429468 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:28.430057 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:28.430765 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:28.430785 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:28.431923 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:28.431928 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:28.433417 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:28.433433 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:28.434049 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:28.434055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:28.434277 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:28.434950 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:28.434968 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:28.436813 (+ 1845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:28.438825 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:28.438836 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:28.438862 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:28.439314 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:28.439633 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:28.439656 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:28.439659 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:28.439880 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:28.440335 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:28.440351 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:28.440752 (+ 401us) client_negotiation.cc:769] Sending connection context
0811 17:19:28.441001 (+ 249us) client_negotiation.cc:240] Negotiation successful
0811 17:19:28.441437 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":141,"threads_started":1}
I20260811 17:19:28.444082 16329 heartbeater.cc:347] Connected to a master server at 127.9.157.190:36801
I20260811 17:19:28.444417 16329 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.130@KRBTEST.COM" created.
I20260811 17:19:28.445145 16329 heartbeater.cc:511] Master 127.9.157.190:36801 requested a full tablet report, sending...
I20260811 17:19:28.446902 16011 ts_manager.cc:194] Registered new tserver with Master: bfeb4cf68dd548feb3e7cb63045bedc9 (127.9.157.129:43555)
I20260811 17:19:28.448649 16011 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.129@KRBTEST.COM'} at 127.9.157.129:39247
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
I20260811 17:19:28.463217 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:0
--local_ip_for_outbound_sockets=127.9.157.130
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:36801
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:38065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:28.585819 16336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:28.586143 16336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:28.586190 16336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:28.590502 16336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:28.590585 16336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:28.590684 16336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:19:28.596359 16336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:38065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:36801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:28.597609 16336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:28.598752 16336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:28.609557 16351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:28.609557 16347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:28.609850 16336 server_base.cc:1034] running on GCE node
W20260811 17:19:28.609561 16344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:28.610399 16336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:28.611336 16336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:28.612553 16336 hybrid_clock.cc:648] HybridClock initialized: now 1786468768612511 us; error 52 us; skew 500 ppm
Aug 11 17:19:28 dist-test-slave-mjjr krb5kdc[15947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468768, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:28.616207 16336 init.cc:377] Logged in from keytab as kudu/127.9.157.130@KRBTEST.COM (short username kudu)
I20260811 17:19:28.617281 16336 webserver.cc:460] Webserver started at http://127.9.157.130:45257/ using document root <none> and password file <none>
I20260811 17:19:28.617942 16336 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:28.618048 16336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:28.618283 16336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:28.620719 16336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-1/data/instance:
uuid: "28b1fa0d84ee4d6ab1527fc6acd0f085"
format_stamp: "Formatted at 2026-08-11 17:19:28 on dist-test-slave-mjjr"
server_key: "8f6a1c15097616b35eeaffdcde4e8911"
server_key_iv: "6311851f4ae3219cb6f0e6522d41ead8"
server_key_version: "encryptionkey@0"
I20260811 17:19:28.621364 16336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance:
uuid: "28b1fa0d84ee4d6ab1527fc6acd0f085"
format_stamp: "Formatted at 2026-08-11 17:19:28 on dist-test-slave-mjjr"
server_key: "8f6a1c15097616b35eeaffdcde4e8911"
server_key_iv: "6311851f4ae3219cb6f0e6522d41ead8"
server_key_version: "encryptionkey@0"
I20260811 17:19:28.625686 16336 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:19:28.628674 16365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:28.630080 16336 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:28.630265 16336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "28b1fa0d84ee4d6ab1527fc6acd0f085"
format_stamp: "Formatted at 2026-08-11 17:19:28 on dist-test-slave-mjjr"
server_key: "8f6a1c15097616b35eeaffdcde4e8911"
server_key_iv: "6311851f4ae3219cb6f0e6522d41ead8"
server_key_version: "encryptionkey@0"
I20260811 17:19:28.630414 16336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:28.641860 16336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:28.646688 16336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:28.646944 16336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:28.647730 16336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:28.648730 16336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:28.648811 16336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:28.648887 16336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:28.648921 16336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:28.659325 16336 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:35017
I20260811 17:19:28.659345 16545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:35017 every 8 connection(s)
I20260811 17:19:28.660490 16336 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:19:28.661929 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 16336
I20260811 17:19:28.662061 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance
I20260811 17:19:28.662369 9846 external_mini_cluster.cc:1428] Setting key a540363f235c3c9974c0d5f6f464a33b
Aug 11 17:19:28 dist-test-slave-mjjr krb5kdc[15947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468768, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.157.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.131@KRBTEST.COM" created.
I20260811 17:19:28.677425 16077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:28.662521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:54893 (local address 127.9.157.190:36801)
0811 17:19:28.662705 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:28.662711 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:28.664224 (+ 1513us) server_negotiation.cc:407] Connection header received
0811 17:19:28.665824 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:28.665828 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:28.665893 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:28.666018 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:28.667389 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:28.668388 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:28.669817 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:28.670295 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:28.670447 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:28.673212 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:28.673220 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:28.673225 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:28.673277 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:28.674900 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:28.675498 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:28.675501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:28.675502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:28.675581 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:28.676020 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:28.676022 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:28.676024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:28.676219 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:28.676300 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:28.677069 (+ 769us) server_negotiation.cc:299] Negotiation successful
0811 17:19:28.677206 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 17:19:28.678301 16549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:28.662972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:36801 (local address 127.9.157.130:54893)
0811 17:19:28.664040 (+ 1068us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:28.664108 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:28.665556 (+ 1448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:28.666230 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:28.666240 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:28.666659 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:28.667218 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:28.667234 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:28.668569 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:28.668573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:28.669656 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:28.669668 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:28.670494 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:28.670497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:28.670662 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:28.671344 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:28.671363 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:28.672990 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:28.675005 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:28.675011 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:28.675027 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:28.675394 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:28.675756 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:28.675759 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:28.675760 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:28.675919 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:28.676377 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:28.676384 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:28.676705 (+ 321us) client_negotiation.cc:769] Sending connection context
0811 17:19:28.676935 (+ 230us) client_negotiation.cc:240] Negotiation successful
0811 17:19:28.677384 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":814,"thread_start_us":120,"threads_started":1}
I20260811 17:19:28.679759 16547 heartbeater.cc:347] Connected to a master server at 127.9.157.190:36801
I20260811 17:19:28.680037 16547 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:28.680639 16547 heartbeater.cc:511] Master 127.9.157.190:36801 requested a full tablet report, sending...
I20260811 17:19:28.681924 16011 ts_manager.cc:194] Registered new tserver with Master: 28b1fa0d84ee4d6ab1527fc6acd0f085 (127.9.157.130:35017)
I20260811 17:19:28.682845 16011 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.130@KRBTEST.COM'} at 127.9.157.130:54893
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
I20260811 17:19:28.695281 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:0
--local_ip_for_outbound_sockets=127.9.157.131
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:36801
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:38065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:28.819757 16553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:28.820151 16553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:28.820248 16553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:28.824724 16553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:28.824834 16553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:28.824939 16553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:19:28.831014 16553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:38065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:36801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:28.832355 16553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:28.833493 16553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:28.844177 16561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:28.844233 16566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:28.844170 16562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:28.844616 16553 server_base.cc:1034] running on GCE node
I20260811 17:19:28.845084 16553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:28.845786 16553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:28.847008 16553 hybrid_clock.cc:648] HybridClock initialized: now 1786468768846969 us; error 49 us; skew 500 ppm
Aug 11 17:19:28 dist-test-slave-mjjr krb5kdc[15947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468768, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:28.850090 16553 init.cc:377] Logged in from keytab as kudu/127.9.157.131@KRBTEST.COM (short username kudu)
I20260811 17:19:28.851071 16553 webserver.cc:460] Webserver started at http://127.9.157.131:35503/ using document root <none> and password file <none>
I20260811 17:19:28.851711 16553 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:28.851780 16553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:28.851979 16553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:28.854126 16553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-2/data/instance:
uuid: "dc6fb9127e8649ffb2a200b2f0ee8bbf"
format_stamp: "Formatted at 2026-08-11 17:19:28 on dist-test-slave-mjjr"
server_key: "ce41b6d0fd5ec8b23d754dde6acaf37e"
server_key_iv: "241426633b2f554e65456ddce83f04c4"
server_key_version: "encryptionkey@0"
I20260811 17:19:28.854691 16553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance:
uuid: "dc6fb9127e8649ffb2a200b2f0ee8bbf"
format_stamp: "Formatted at 2026-08-11 17:19:28 on dist-test-slave-mjjr"
server_key: "ce41b6d0fd5ec8b23d754dde6acaf37e"
server_key_iv: "241426633b2f554e65456ddce83f04c4"
server_key_version: "encryptionkey@0"
I20260811 17:19:28.858896 16553 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 17:19:28.861840 16582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:28.863204 16553 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:19:28.863317 16553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "dc6fb9127e8649ffb2a200b2f0ee8bbf"
format_stamp: "Formatted at 2026-08-11 17:19:28 on dist-test-slave-mjjr"
server_key: "ce41b6d0fd5ec8b23d754dde6acaf37e"
server_key_iv: "241426633b2f554e65456ddce83f04c4"
server_key_version: "encryptionkey@0"
I20260811 17:19:28.863472 16553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:28.885493 16553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:28.889210 16553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:28.889451 16553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:28.890151 16553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:28.891086 16553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:28.891160 16553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:28.891230 16553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:28.891273 16553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:28.901328 16553 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:35097
I20260811 17:19:28.901356 16763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:35097 every 8 connection(s)
I20260811 17:19:28.902341 16553 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:19:28.904228 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 16553
I20260811 17:19:28.904392 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance
I20260811 17:19:28.904747 9846 external_mini_cluster.cc:1428] Setting key e46b9cfad774e298175f67f440e0d954
Aug 11 17:19:28 dist-test-slave-mjjr krb5kdc[15947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468768, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:28.919204 16077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:28.904416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:33233 (local address 127.9.157.190:36801)
0811 17:19:28.904588 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:28.904594 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:28.905618 (+ 1024us) server_negotiation.cc:407] Connection header received
0811 17:19:28.907694 (+ 2076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:28.907697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:28.907748 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:28.907821 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:28.909126 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:28.909695 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:28.911119 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:28.911347 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:28.911547 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:28.914455 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:28.914459 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:28.914462 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:28.914493 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:28.916100 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:28.916732 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:28.916737 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:28.916739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:28.916841 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:28.917363 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:28.917367 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:28.917369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:28.917648 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:28.917764 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:28.918540 (+ 776us) server_negotiation.cc:299] Negotiation successful
0811 17:19:28.919058 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 17:19:28.920419 16766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:28.904856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:36801 (local address 127.9.157.131:33233)
0811 17:19:28.905441 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:28.905515 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:28.907381 (+ 1866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:28.908024 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:28.908032 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:28.908424 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:28.908966 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:28.908980 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:28.909846 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:28.909849 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:28.910957 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:28.910974 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:28.911627 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:28.911631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:28.911840 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:28.912450 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:28.912469 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:28.914224 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:28.916259 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:28.916267 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:28.916294 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:28.916599 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:28.917058 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:28.917064 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:28.917070 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:28.917246 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:28.917803 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:28.917810 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:28.918249 (+ 439us) client_negotiation.cc:769] Sending connection context
0811 17:19:28.918498 (+ 249us) client_negotiation.cc:240] Negotiation successful
0811 17:19:28.918881 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":127,"threads_started":1}
I20260811 17:19:28.921660 16764 heartbeater.cc:347] Connected to a master server at 127.9.157.190:36801
I20260811 17:19:28.921911 16764 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:28.922434 16764 heartbeater.cc:511] Master 127.9.157.190:36801 requested a full tablet report, sending...
I20260811 17:19:28.923573 16011 ts_manager.cc:194] Registered new tserver with Master: dc6fb9127e8649ffb2a200b2f0ee8bbf (127.9.157.131:35097)
I20260811 17:19:28.924252 16011 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.131@KRBTEST.COM'} at 127.9.157.131:33233
I20260811 17:19:28.932600 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 17:19:28 dist-test-slave-mjjr krb5kdc[15947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 17:19:28 dist-test-slave-mjjr krb5kdc[15947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.129@KRBTEST.COM
I20260811 17:19:28.961920 16772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:28.951580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41894 (local address 127.9.157.129:43555)
0811 17:19:28.951968 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:28.951974 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:28.952002 (+ 28us) server_negotiation.cc:407] Connection header received
0811 17:19:28.952056 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:28.952061 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:28.952285 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:28.952422 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:28.953175 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:28.953923 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:28.954803 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:28.955173 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:28.955253 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:28.958195 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:28.958206 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:28.958222 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:28.958249 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:28.960072 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:28.960510 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:28.960516 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:28.960517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:28.960596 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:28.960850 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:28.960852 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:28.960853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:28.961103 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:28.961263 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:28.961540 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 17:19:28.961669 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":190,"threads_started":1}
Aug 11 17:19:28 dist-test-slave-mjjr krb5kdc[15947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:28.973796 16077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:28.965122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34680 (local address 127.9.157.190:36801)
0811 17:19:28.965282 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:28.965288 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:28.965370 (+ 82us) server_negotiation.cc:407] Connection header received
0811 17:19:28.965539 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:28.965542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:28.965586 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:28.965672 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:28.966301 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:28.966845 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:28.967799 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:28.968024 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:28.968141 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:28.970638 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:28.970641 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:28.970644 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:28.970671 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:28.972005 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:28.972409 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:28.972411 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:28.972412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:28.972484 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:28.972768 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:28.972771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:28.972772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:28.973046 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:28.973167 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:28.973486 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 17:19:28.973592 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 17:19:28.974335 16011 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34680
I20260811 17:19:28.975241 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 16111
I20260811 17:19:28.989852 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 16336
I20260811 17:19:29.006281 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 16553
I20260811 17:19:29.022853 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 15968
2026-08-11T17:19:29Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2562 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:19:29 dist-test-slave-mjjr krb5kdc[16780](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:19:29 dist-test-slave-mjjr krb5kdc[16780](info): set up 2 sockets
Aug 11 17:19:29 dist-test-slave-mjjr krb5kdc[16780](info): commencing operation
krb5kdc: starting...
W20260811 17:19:30.319114 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.263s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:19:30 dist-test-slave-mjjr krb5kdc[16780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:19:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:19:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:19:30.458339 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:44719
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:39007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:44719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:30.580806 16795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:30.581171 16795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:30.581251 16795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:30.585704 16795 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:19:30.585808 16795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:30.585846 16795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:30.585907 16795 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:19:30.585945 16795 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:19:30.591862 16795 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:39007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:44719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:44719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:30.593241 16795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:30.594455 16795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:30.604648 16807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:30.604773 16803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:30.604822 16805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:30.605470 16795 server_base.cc:1034] running on GCE node
I20260811 17:19:30.606218 16795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:30.607529 16795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:30.608755 16795 hybrid_clock.cc:648] HybridClock initialized: now 1786468770608719 us; error 62 us; skew 500 ppm
Aug 11 17:19:30 dist-test-slave-mjjr krb5kdc[16780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468770, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:30.615839 16795 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:19:30.616744 16795 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:30.616827 16795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:30.617017 16795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:30.619236 16795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "1361af3f3396456787fc3f95a62d3237"
format_stamp: "Formatted at 2026-08-11 17:19:30 on dist-test-slave-mjjr"
server_key: "ac27da49d0cf15fb80fdb1689ca5666f"
server_key_iv: "3caa9637afcae0ee2d831fa4827c8638"
server_key_version: "encryptionkey@0"
I20260811 17:19:30.619889 16795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "1361af3f3396456787fc3f95a62d3237"
format_stamp: "Formatted at 2026-08-11 17:19:30 on dist-test-slave-mjjr"
server_key: "ac27da49d0cf15fb80fdb1689ca5666f"
server_key_iv: "3caa9637afcae0ee2d831fa4827c8638"
server_key_version: "encryptionkey@0"
I20260811 17:19:30.623931 16795 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:19:30.626600 16819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:30.628090 16795 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:30.628247 16795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "1361af3f3396456787fc3f95a62d3237"
format_stamp: "Formatted at 2026-08-11 17:19:30 on dist-test-slave-mjjr"
server_key: "ac27da49d0cf15fb80fdb1689ca5666f"
server_key_iv: "3caa9637afcae0ee2d831fa4827c8638"
server_key_version: "encryptionkey@0"
I20260811 17:19:30.628368 16795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:30.639287 16795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:30.643326 16795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:30.643610 16795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:30.655911 16795 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:44719
I20260811 17:19:30.656177 16899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:44719 every 8 connection(s)
I20260811 17:19:30.657153 16795 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:19:30.660392 16903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:30.666860 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 16795
I20260811 17:19:30.666929 16903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1361af3f3396456787fc3f95a62d3237: Bootstrap starting.
I20260811 17:19:30.667060 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:19:30.667410 9846 external_mini_cluster.cc:1428] Setting key 860df063fae53fd1aad79b42b68f4c45
I20260811 17:19:30.670074 16903 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1361af3f3396456787fc3f95a62d3237: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:30.670950 16903 log.cc:824] T 00000000000000000000000000000000 P 1361af3f3396456787fc3f95a62d3237: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:30.673571 16903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1361af3f3396456787fc3f95a62d3237: No bootstrap required, opened a new log
Aug 11 17:19:30 dist-test-slave-mjjr krb5kdc[16780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:30.678725 16903 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1361af3f3396456787fc3f95a62d3237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1361af3f3396456787fc3f95a62d3237" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44719 } }
I20260811 17:19:30.679008 16903 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1361af3f3396456787fc3f95a62d3237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:30.679093 16903 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1361af3f3396456787fc3f95a62d3237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1361af3f3396456787fc3f95a62d3237, State: Initialized, Role: FOLLOWER
I20260811 17:19:30.679561 16903 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1361af3f3396456787fc3f95a62d3237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1361af3f3396456787fc3f95a62d3237" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44719 } }
I20260811 17:19:30.679780 16903 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1361af3f3396456787fc3f95a62d3237 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:19:30.679875 16903 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1361af3f3396456787fc3f95a62d3237 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:19:30.679998 16903 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1361af3f3396456787fc3f95a62d3237 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:30.684939 16910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:30.669075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39560 (local address 127.9.157.190:44719)
0811 17:19:30.669687 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:30.669708 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:30.669752 (+ 44us) server_negotiation.cc:407] Connection header received
0811 17:19:30.670698 (+ 946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:30.670721 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:30.671241 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:30.671760 (+ 519us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:30.672842 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:30.674148 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:30.675414 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:30.676034 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:30.676196 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:30.679684 (+ 3488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:30.679697 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:30.679715 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:30.679749 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:30.681780 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:30.682282 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:30.682301 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:30.682306 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:30.682426 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:30.682738 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:30.682742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:30.682743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:30.683185 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:30.683403 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:30.683748 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 17:19:30.684081 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":163,"threads_started":1}
I20260811 17:19:30.743287 16903 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1361af3f3396456787fc3f95a62d3237 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1361af3f3396456787fc3f95a62d3237" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44719 } }
I20260811 17:19:30.743920 16903 leader_election.cc:302] T 00000000000000000000000000000000 P 1361af3f3396456787fc3f95a62d3237 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1361af3f3396456787fc3f95a62d3237; no voters:
I20260811 17:19:30.744354 16903 leader_election.cc:288] T 00000000000000000000000000000000 P 1361af3f3396456787fc3f95a62d3237 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:19:30.744517 16913 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1361af3f3396456787fc3f95a62d3237 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:30.744917 16913 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1361af3f3396456787fc3f95a62d3237 [term 1 LEADER]: Becoming Leader. State: Replica: 1361af3f3396456787fc3f95a62d3237, State: Running, Role: LEADER
I20260811 17:19:30.745501 16913 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1361af3f3396456787fc3f95a62d3237 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1361af3f3396456787fc3f95a62d3237" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44719 } }
I20260811 17:19:30.746021 16903 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1361af3f3396456787fc3f95a62d3237 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:19:30.747777 16915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1361af3f3396456787fc3f95a62d3237 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1361af3f3396456787fc3f95a62d3237. Latest consensus state: current_term: 1 leader_uuid: "1361af3f3396456787fc3f95a62d3237" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1361af3f3396456787fc3f95a62d3237" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44719 } } }
I20260811 17:19:30.747941 16915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1361af3f3396456787fc3f95a62d3237 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:30.748343 16914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1361af3f3396456787fc3f95a62d3237 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1361af3f3396456787fc3f95a62d3237" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1361af3f3396456787fc3f95a62d3237" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44719 } } }
I20260811 17:19:30.748685 16914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1361af3f3396456787fc3f95a62d3237 [sys.catalog]: This master's current role is: LEADER
W20260811 17:19:30.751909 16932 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1361af3f3396456787fc3f95a62d3237: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:19:30.752034 16932 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:19:30.752115 16930 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:19:30.753201 16930 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:19:30.759104 16930 catalog_manager.cc:1347] Generated new cluster ID: bfcd88a8d2a149aaaaad8c79b7d756f6
I20260811 17:19:30.759193 16930 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:19:30.768831 16930 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:19:30.769847 16930 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:19:30.774966 16930 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1361af3f3396456787fc3f95a62d3237: Generated new TSK 0
I20260811 17:19:30.775822 16930 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
W20260811 17:19:31.647732 9846 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.9.157.129: real 0.863s user 0.002s sys 0.003s
I20260811 17:19:31.648351 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:0
--local_ip_for_outbound_sockets=127.9.157.129
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:44719
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:39007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:31.775556 16940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:31.775971 16940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:31.776029 16940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:31.780371 16940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:31.780447 16940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:31.780512 16940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:19:31.786185 16940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:39007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:44719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:31.787421 16940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:31.788590 16940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:31.799189 16952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:31.799190 16949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:31.799394 16940 server_base.cc:1034] running on GCE node
W20260811 17:19:31.799190 16950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:31.800101 16940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:31.800887 16940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:31.802095 16940 hybrid_clock.cc:648] HybridClock initialized: now 1786468771802055 us; error 51 us; skew 500 ppm
Aug 11 17:19:31 dist-test-slave-mjjr krb5kdc[16780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468771, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:31.805302 16940 init.cc:377] Logged in from keytab as kudu/127.9.157.129@KRBTEST.COM (short username kudu)
I20260811 17:19:31.806259 16940 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:31.806371 16940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:31.806666 16940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:31.809197 16940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-0/data/instance:
uuid: "281a2fe30fd14636bba35e6d0215694a"
format_stamp: "Formatted at 2026-08-11 17:19:31 on dist-test-slave-mjjr"
server_key: "fca10d9514720692e326b5757d484329"
server_key_iv: "6fa7cbf9679bcf6433a51e928007e258"
server_key_version: "encryptionkey@0"
I20260811 17:19:31.809904 16940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance:
uuid: "281a2fe30fd14636bba35e6d0215694a"
format_stamp: "Formatted at 2026-08-11 17:19:31 on dist-test-slave-mjjr"
server_key: "fca10d9514720692e326b5757d484329"
server_key_iv: "6fa7cbf9679bcf6433a51e928007e258"
server_key_version: "encryptionkey@0"
I20260811 17:19:31.815081 16940 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 17:19:31.818434 16959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:31.820055 16940 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:31.820223 16940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "281a2fe30fd14636bba35e6d0215694a"
format_stamp: "Formatted at 2026-08-11 17:19:31 on dist-test-slave-mjjr"
server_key: "fca10d9514720692e326b5757d484329"
server_key_iv: "6fa7cbf9679bcf6433a51e928007e258"
server_key_version: "encryptionkey@0"
I20260811 17:19:31.820367 16940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:31.843631 16940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:31.847263 16940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:31.847533 16940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:31.848290 16940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:31.849220 16940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:31.849272 16940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:31.849313 16940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:31.849366 16940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:31.862064 16940 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:39363
I20260811 17:19:31.862087 17152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:39363 every 8 connection(s)
I20260811 17:19:31.862792 16940 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:19:31.868000 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 16940
I20260811 17:19:31.868178 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance
I20260811 17:19:31.868561 9846 external_mini_cluster.cc:1428] Setting key d68b27bf3e582cb8c90c9f5f57626903
Aug 11 17:19:31 dist-test-slave-mjjr krb5kdc[16780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468771, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:31.882710 17157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:31.865130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:43155 (local address 127.9.157.190:44719)
0811 17:19:31.869226 (+ 4096us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:31.869241 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:31.869262 (+ 21us) server_negotiation.cc:407] Connection header received
0811 17:19:31.869316 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:31.869319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:31.869384 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:31.869494 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:31.871537 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:31.872423 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:31.874354 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:31.874792 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:31.874940 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:31.877914 (+ 2974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:31.877921 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:31.877928 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:31.877968 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:31.880122 (+ 2154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:31.880733 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:31.880739 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:31.880742 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:31.880887 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:31.881269 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:31.881273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:31.881275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:31.881580 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:31.881714 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:31.882296 (+ 582us) server_negotiation.cc:299] Negotiation successful
0811 17:19:31.882463 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3985,"thread_start_us":171,"threads_started":1}
WARNING: no policy specified for kudu/127.9.157.130@KRBTEST.COM; defaulting to no policy
I20260811 17:19:31.883544 17159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:31.865489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44719 (local address 127.9.157.129:43155)
0811 17:19:31.866304 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:31.866397 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:31.868355 (+ 1958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:31.869815 (+ 1460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:31.869826 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:31.870489 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:31.871344 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:31.871372 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:31.872590 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:31.872597 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:31.874171 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:31.874189 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:31.875023 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:31.875027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:31.875194 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:31.875963 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:31.875982 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:31.877640 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:31.880293 (+ 2653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:31.880299 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:31.880317 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:31.880611 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:31.881007 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:31.881009 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:31.881011 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:31.881156 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:31.881697 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:31.881703 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:31.881997 (+ 294us) client_negotiation.cc:769] Sending connection context
0811 17:19:31.882175 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 17:19:31.882488 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":126,"threads_started":1}
I20260811 17:19:31.885159 17154 heartbeater.cc:347] Connected to a master server at 127.9.157.190:44719
I20260811 17:19:31.885447 17154 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:31.885973 17154 heartbeater.cc:511] Master 127.9.157.190:44719 requested a full tablet report, sending...
I20260811 17:19:31.887742 16843 ts_manager.cc:194] Registered new tserver with Master: 281a2fe30fd14636bba35e6d0215694a (127.9.157.129:39363)
I20260811 17:19:31.888970 16843 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.129@KRBTEST.COM'} at 127.9.157.129:43155
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
W20260811 17:19:32.418825 9846 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.9.157.130: real 0.547s user 0.005s sys 0.000s
I20260811 17:19:32.419371 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:0
--local_ip_for_outbound_sockets=127.9.157.130
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:44719
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:39007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:32.539618 17164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:32.539980 17164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:32.540059 17164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:32.544299 17164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:32.544406 17164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:32.544512 17164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:19:32.550495 17164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:39007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:44719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:32.551819 17164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:32.552950 17164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:32.563462 17169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:32.563462 17172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:32.563520 17170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:32.563956 17164 server_base.cc:1034] running on GCE node
I20260811 17:19:32.564443 17164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:32.565145 17164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:32.566386 17164 hybrid_clock.cc:648] HybridClock initialized: now 1786468772566369 us; error 39 us; skew 500 ppm
Aug 11 17:19:32 dist-test-slave-mjjr krb5kdc[16780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468772, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:32.569788 17164 init.cc:377] Logged in from keytab as kudu/127.9.157.130@KRBTEST.COM (short username kudu)
I20260811 17:19:32.570695 17164 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:32.570811 17164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:32.571095 17164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:32.573529 17164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-1/data/instance:
uuid: "71aef0a85bfe4be5a6113d9918f1c2b6"
format_stamp: "Formatted at 2026-08-11 17:19:32 on dist-test-slave-mjjr"
server_key: "8fc39ec2381ce5ab841e6cbf07ff79ac"
server_key_iv: "aedc473e6348264b73395e50bbd46900"
server_key_version: "encryptionkey@0"
I20260811 17:19:32.574216 17164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance:
uuid: "71aef0a85bfe4be5a6113d9918f1c2b6"
format_stamp: "Formatted at 2026-08-11 17:19:32 on dist-test-slave-mjjr"
server_key: "8fc39ec2381ce5ab841e6cbf07ff79ac"
server_key_iv: "aedc473e6348264b73395e50bbd46900"
server_key_version: "encryptionkey@0"
I20260811 17:19:32.578982 17164 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 17:19:32.581900 17186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:32.583387 17164 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:19:32.583568 17164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "71aef0a85bfe4be5a6113d9918f1c2b6"
format_stamp: "Formatted at 2026-08-11 17:19:32 on dist-test-slave-mjjr"
server_key: "8fc39ec2381ce5ab841e6cbf07ff79ac"
server_key_iv: "aedc473e6348264b73395e50bbd46900"
server_key_version: "encryptionkey@0"
I20260811 17:19:32.583767 17164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:32.603353 17164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:32.607215 17164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:32.607475 17164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:32.608237 17164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:32.609206 17164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:32.609292 17164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:32.609367 17164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:32.609419 17164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:32.619614 17164 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:45411
I20260811 17:19:32.619700 17366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:45411 every 8 connection(s)
I20260811 17:19:32.620373 17164 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:19:32.628068 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 17164
I20260811 17:19:32.628242 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance
I20260811 17:19:32.628573 9846 external_mini_cluster.cc:1428] Setting key a5e9b4e81236cf81ae3446952dd55386
Aug 11 17:19:32 dist-test-slave-mjjr krb5kdc[16780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468772, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:32.636847 17371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:32.622431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:46641 (local address 127.9.157.190:44719)
0811 17:19:32.622717 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:32.622723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:32.623776 (+ 1053us) server_negotiation.cc:407] Connection header received
0811 17:19:32.625332 (+ 1556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:32.625344 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:32.625419 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:32.625539 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:32.626908 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:32.627482 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:32.629254 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:32.629510 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:32.629636 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:32.632641 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:32.632645 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:32.632647 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:32.632677 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:32.634367 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:32.634960 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:32.634964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:32.634965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:32.635053 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:32.635440 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:32.635444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:32.635445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:32.635732 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:32.635854 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:32.636502 (+ 648us) server_negotiation.cc:299] Negotiation successful
0811 17:19:32.636647 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":67,"threads_started":1}
I20260811 17:19:32.637738 17372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:32.622954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44719 (local address 127.9.157.130:46641)
0811 17:19:32.623524 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:32.623572 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:32.625087 (+ 1515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:32.625742 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:32.625750 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:32.626189 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:32.626738 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:32.626752 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:32.627636 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:32.627682 (+ 46us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:32.629114 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:32.629125 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:32.629689 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:32.629694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:32.629884 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:32.630562 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:32.630581 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:32.632438 (+ 1857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:32.634509 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:32.634516 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:32.634538 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:32.634831 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:32.635153 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:32.635156 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:32.635159 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:32.635322 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:32.635846 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:32.635854 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:32.636210 (+ 356us) client_negotiation.cc:769] Sending connection context
0811 17:19:32.636442 (+ 232us) client_negotiation.cc:240] Negotiation successful
0811 17:19:32.636813 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":136,"threads_started":1}
I20260811 17:19:32.639070 17368 heartbeater.cc:347] Connected to a master server at 127.9.157.190:44719
I20260811 17:19:32.639362 17368 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:32.639941 17368 heartbeater.cc:511] Master 127.9.157.190:44719 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.157.131@KRBTEST.COM; defaulting to no policy
I20260811 17:19:32.641137 16843 ts_manager.cc:194] Registered new tserver with Master: 71aef0a85bfe4be5a6113d9918f1c2b6 (127.9.157.130:45411)
I20260811 17:19:32.641775 16843 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.130@KRBTEST.COM'} at 127.9.157.130:46641
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
I20260811 17:19:32.663108 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:0
--local_ip_for_outbound_sockets=127.9.157.131
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:44719
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:39007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:32.787345 17375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:32.787703 17375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:32.787756 17375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:32.792317 17375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:32.792400 17375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:32.792466 17375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:19:32.798347 17375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:39007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:44719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:32.799657 17375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:32.800930 17375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:19:32.811714 17375 server_base.cc:1034] running on GCE node
W20260811 17:19:32.811658 17384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:32.811658 17385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:32.811661 17389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:32.812500 17375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:32.813299 17375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:32.814502 17375 hybrid_clock.cc:648] HybridClock initialized: now 1786468772814459 us; error 53 us; skew 500 ppm
Aug 11 17:19:32 dist-test-slave-mjjr krb5kdc[16780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468772, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:32.818138 17375 init.cc:377] Logged in from keytab as kudu/127.9.157.131@KRBTEST.COM (short username kudu)
I20260811 17:19:32.819096 17375 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:32.819180 17375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:32.819406 17375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:32.821928 17375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-2/data/instance:
uuid: "f542f3f36be44a2d9d93f9dd27051b46"
format_stamp: "Formatted at 2026-08-11 17:19:32 on dist-test-slave-mjjr"
server_key: "6ac7470ebdf2634b29c68c2a0257d13d"
server_key_iv: "e8798bfd6a4a97b74fd2ad98597cb602"
server_key_version: "encryptionkey@0"
I20260811 17:19:32.822522 17375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance:
uuid: "f542f3f36be44a2d9d93f9dd27051b46"
format_stamp: "Formatted at 2026-08-11 17:19:32 on dist-test-slave-mjjr"
server_key: "6ac7470ebdf2634b29c68c2a0257d13d"
server_key_iv: "e8798bfd6a4a97b74fd2ad98597cb602"
server_key_version: "encryptionkey@0"
I20260811 17:19:32.827024 17375 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:19:32.829908 17398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:32.831409 17375 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:19:32.831557 17375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "f542f3f36be44a2d9d93f9dd27051b46"
format_stamp: "Formatted at 2026-08-11 17:19:32 on dist-test-slave-mjjr"
server_key: "6ac7470ebdf2634b29c68c2a0257d13d"
server_key_iv: "e8798bfd6a4a97b74fd2ad98597cb602"
server_key_version: "encryptionkey@0"
I20260811 17:19:32.831755 17375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:32.846750 17375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:32.850544 17375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:32.850770 17375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:32.851481 17375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:32.852460 17375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:32.852511 17375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:32.852586 17375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:32.852619 17375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:32.862862 17375 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:33239
I20260811 17:19:32.862890 17583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:33239 every 8 connection(s)
I20260811 17:19:32.863559 17375 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:19:32.872365 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 17375
I20260811 17:19:32.872520 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance
I20260811 17:19:32.872889 9846 external_mini_cluster.cc:1428] Setting key 40ed6d2497d8496103eca600287dfb17
Aug 11 17:19:32 dist-test-slave-mjjr krb5kdc[16780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468772, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:32.879460 17371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:32.865821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:47689 (local address 127.9.157.190:44719)
0811 17:19:32.866003 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:32.866009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:32.867481 (+ 1472us) server_negotiation.cc:407] Connection header received
0811 17:19:32.868485 (+ 1004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:32.868488 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:32.868548 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:32.868637 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:32.870111 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:32.870682 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:32.872132 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:32.872524 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:32.872647 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:32.875757 (+ 3110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:32.875761 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:32.875763 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:32.875791 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:32.877181 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:32.877786 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:32.877789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:32.877790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:32.877912 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:32.878285 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:32.878288 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:32.878289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:32.878494 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:32.878595 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:32.879200 (+ 605us) server_negotiation.cc:299] Negotiation successful
0811 17:19:32.879327 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 17:19:32.880436 17588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:32.866209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44719 (local address 127.9.157.131:47689)
0811 17:19:32.867249 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:32.867310 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:32.868206 (+ 896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:32.868913 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:32.868921 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:32.869338 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:32.869899 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:32.869916 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:32.870857 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:32.870861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:32.871963 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:32.871974 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:32.872726 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:32.872729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:32.872895 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:32.873644 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:32.873668 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:32.875474 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:32.877326 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:32.877332 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:32.877348 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:32.877663 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:32.878017 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:32.878019 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:32.878021 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:32.878183 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:32.878660 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:32.878666 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:32.878954 (+ 288us) client_negotiation.cc:769] Sending connection context
0811 17:19:32.879139 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 17:19:32.879447 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":186,"threads_started":1}
I20260811 17:19:32.881726 17585 heartbeater.cc:347] Connected to a master server at 127.9.157.190:44719
I20260811 17:19:32.882061 17585 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:32.882606 17585 heartbeater.cc:511] Master 127.9.157.190:44719 requested a full tablet report, sending...
I20260811 17:19:32.883945 16843 ts_manager.cc:194] Registered new tserver with Master: f542f3f36be44a2d9d93f9dd27051b46 (127.9.157.131:33239)
I20260811 17:19:32.884572 16843 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.131@KRBTEST.COM'} at 127.9.157.131:47689
I20260811 17:19:32.887706 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:19:32.891615 17154 heartbeater.cc:503] Master 127.9.157.190:44719 was elected leader, sending a full tablet report...
I20260811 17:19:32.897768 17371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:32.889823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39564 (local address 127.9.157.190:44719)
0811 17:19:32.889986 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:32.889992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:32.890225 (+ 233us) server_negotiation.cc:407] Connection header received
0811 17:19:32.890343 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:32.890346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:32.890389 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:32.890461 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:32.891240 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:32.891881 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:32.893030 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:32.893252 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:32.893332 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:32.894658 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:32.894662 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:32.894668 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:32.894694 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:32.896271 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:32.896686 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:32.896689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:32.896690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:32.896765 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:32.897003 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:32.897005 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:32.897006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:32.897226 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:32.897335 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:32.897545 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 17:19:32.897643 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 17:19:32.901168 16843 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:19:32.904491 16843 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:19:32.921646 17602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:32.914527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:45411 (local address 127.0.0.1:43820)
0811 17:19:32.915487 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:32.915502 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:32.915593 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:32.916556 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:32.916559 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:32.916584 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:32.916815 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:32.916825 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:32.918544 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:32.918550 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:32.920091 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:32.920102 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:32.921004 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:32.921008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:32.921133 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:32.921177 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 17:19:32.921303 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 17:19:32.921454 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":882,"mutex_wait_us":133,"spinlock_wait_cycles":1408,"thread_start_us":115,"threads_started":1}
I20260811 17:19:32.921746 17608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:32.915855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35540 (local address 127.9.157.131:33239)
0811 17:19:32.916579 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:32.916605 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:32.916630 (+ 25us) server_negotiation.cc:407] Connection header received
0811 17:19:32.916715 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:32.916720 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:32.916942 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:32.917119 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:32.917303 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:32.918086 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:32.920148 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:32.921036 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:32.921246 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:32.921411 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:32.921471 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 17:19:32.921563 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":158,"threads_started":1}
I20260811 17:19:32.921646 17605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:32.915361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:33239 (local address 127.0.0.1:35540)
0811 17:19:32.916167 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:32.916183 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:32.916281 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:32.917091 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:32.917094 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:32.917114 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:32.917202 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:32.917208 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:32.918230 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:32.918235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:32.919981 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:32.919994 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:32.921191 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:32.921196 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:32.921335 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:32.921366 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 17:19:32.921424 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 17:19:32.921479 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"mutex_wait_us":201,"thread_start_us":101,"threads_started":1}
I20260811 17:19:32.922333 17603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:32.914500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:39363 (local address 127.0.0.1:53662)
0811 17:19:32.914984 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:32.915046 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:32.915204 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:32.916471 (+ 1267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:32.916481 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:32.916513 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:32.916829 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:32.916837 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:32.918461 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:32.918464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:32.919981 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:32.919994 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:32.921191 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:32.921196 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:32.921759 (+ 563us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:32.921787 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 17:19:32.921865 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 17:19:32.921938 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":637,"spinlock_wait_cycles":1792,"thread_start_us":142,"threads_started":1}
I20260811 17:19:32.922399 17607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:32.915597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43820 (local address 127.9.157.130:45411)
0811 17:19:32.916103 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:32.916116 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:32.916139 (+ 23us) server_negotiation.cc:407] Connection header received
0811 17:19:32.916216 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:32.916221 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:32.916396 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:32.917116 (+ 720us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:32.917299 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:32.918423 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:32.920234 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:32.920860 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:32.920999 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:32.921178 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:32.922137 (+ 959us) server_negotiation.cc:299] Negotiation successful
0811 17:19:32.922238 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":182,"threads_started":1}
I20260811 17:19:32.922302 17604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:32.914701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53662 (local address 127.9.157.129:39363)
0811 17:19:32.915843 (+ 1142us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:32.915878 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:32.915910 (+ 32us) server_negotiation.cc:407] Connection header received
0811 17:19:32.916006 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:32.916012 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:32.916260 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:32.916748 (+ 488us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:32.917015 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:32.918329 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:32.920149 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:32.921037 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:32.921269 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:32.921411 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:32.922037 (+ 626us) server_negotiation.cc:299] Negotiation successful
0811 17:19:32.922148 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1027,"thread_start_us":142,"threads_started":1}
I20260811 17:19:32.925647 17261 tablet_service.cc:1467] Processing CreateTablet for tablet 974297131f7f4fde92b33980ca2812af (DEFAULT_TABLE table=test-table [id=b597e913130647ac967b7c57c4624d40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:32.925625 17040 tablet_service.cc:1467] Processing CreateTablet for tablet 974297131f7f4fde92b33980ca2812af (DEFAULT_TABLE table=test-table [id=b597e913130647ac967b7c57c4624d40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:32.925639 17471 tablet_service.cc:1467] Processing CreateTablet for tablet 974297131f7f4fde92b33980ca2812af (DEFAULT_TABLE table=test-table [id=b597e913130647ac967b7c57c4624d40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:32.926975 17261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 974297131f7f4fde92b33980ca2812af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:32.926965 17040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 974297131f7f4fde92b33980ca2812af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:32.926980 17471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 974297131f7f4fde92b33980ca2812af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:32.935132 17609 tablet_bootstrap.cc:492] T 974297131f7f4fde92b33980ca2812af P 71aef0a85bfe4be5a6113d9918f1c2b6: Bootstrap starting.
I20260811 17:19:32.936800 17614 tablet_bootstrap.cc:492] T 974297131f7f4fde92b33980ca2812af P f542f3f36be44a2d9d93f9dd27051b46: Bootstrap starting.
I20260811 17:19:32.937837 17609 tablet_bootstrap.cc:654] T 974297131f7f4fde92b33980ca2812af P 71aef0a85bfe4be5a6113d9918f1c2b6: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:32.938802 17614 tablet_bootstrap.cc:654] T 974297131f7f4fde92b33980ca2812af P f542f3f36be44a2d9d93f9dd27051b46: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:32.939152 17609 log.cc:824] T 974297131f7f4fde92b33980ca2812af P 71aef0a85bfe4be5a6113d9918f1c2b6: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:32.940114 17614 log.cc:824] T 974297131f7f4fde92b33980ca2812af P f542f3f36be44a2d9d93f9dd27051b46: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:32.940119 17610 tablet_bootstrap.cc:492] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a: Bootstrap starting.
I20260811 17:19:32.941721 17609 tablet_bootstrap.cc:492] T 974297131f7f4fde92b33980ca2812af P 71aef0a85bfe4be5a6113d9918f1c2b6: No bootstrap required, opened a new log
I20260811 17:19:32.941972 17609 ts_tablet_manager.cc:1397] T 974297131f7f4fde92b33980ca2812af P 71aef0a85bfe4be5a6113d9918f1c2b6: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 17:19:32.942371 17614 tablet_bootstrap.cc:492] T 974297131f7f4fde92b33980ca2812af P f542f3f36be44a2d9d93f9dd27051b46: No bootstrap required, opened a new log
I20260811 17:19:32.942466 17610 tablet_bootstrap.cc:654] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:32.942559 17614 ts_tablet_manager.cc:1397] T 974297131f7f4fde92b33980ca2812af P f542f3f36be44a2d9d93f9dd27051b46: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 17:19:32.943176 17610 log.cc:824] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:32.944972 17610 tablet_bootstrap.cc:492] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a: No bootstrap required, opened a new log
I20260811 17:19:32.945190 17610 ts_tablet_manager.cc:1397] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 17:19:32.946336 17609 raft_consensus.cc:348] T 974297131f7f4fde92b33980ca2812af P 71aef0a85bfe4be5a6113d9918f1c2b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "281a2fe30fd14636bba35e6d0215694a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39363 } } peers { permanent_uuid: "71aef0a85bfe4be5a6113d9918f1c2b6" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 45411 } } peers { permanent_uuid: "f542f3f36be44a2d9d93f9dd27051b46" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 33239 } }
I20260811 17:19:32.946700 17609 raft_consensus.cc:374] T 974297131f7f4fde92b33980ca2812af P 71aef0a85bfe4be5a6113d9918f1c2b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:32.946753 17609 raft_consensus.cc:729] T 974297131f7f4fde92b33980ca2812af P 71aef0a85bfe4be5a6113d9918f1c2b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71aef0a85bfe4be5a6113d9918f1c2b6, State: Initialized, Role: FOLLOWER
I20260811 17:19:32.946591 17614 raft_consensus.cc:348] T 974297131f7f4fde92b33980ca2812af P f542f3f36be44a2d9d93f9dd27051b46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "281a2fe30fd14636bba35e6d0215694a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39363 } } peers { permanent_uuid: "71aef0a85bfe4be5a6113d9918f1c2b6" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 45411 } } peers { permanent_uuid: "f542f3f36be44a2d9d93f9dd27051b46" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 33239 } }
I20260811 17:19:32.946856 17614 raft_consensus.cc:374] T 974297131f7f4fde92b33980ca2812af P f542f3f36be44a2d9d93f9dd27051b46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:32.946902 17614 raft_consensus.cc:729] T 974297131f7f4fde92b33980ca2812af P f542f3f36be44a2d9d93f9dd27051b46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f542f3f36be44a2d9d93f9dd27051b46, State: Initialized, Role: FOLLOWER
I20260811 17:19:32.947292 17609 consensus_queue.cc:260] T 974297131f7f4fde92b33980ca2812af P 71aef0a85bfe4be5a6113d9918f1c2b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "281a2fe30fd14636bba35e6d0215694a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39363 } } peers { permanent_uuid: "71aef0a85bfe4be5a6113d9918f1c2b6" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 45411 } } peers { permanent_uuid: "f542f3f36be44a2d9d93f9dd27051b46" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 33239 } }
I20260811 17:19:32.947304 17614 consensus_queue.cc:260] T 974297131f7f4fde92b33980ca2812af P f542f3f36be44a2d9d93f9dd27051b46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "281a2fe30fd14636bba35e6d0215694a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39363 } } peers { permanent_uuid: "71aef0a85bfe4be5a6113d9918f1c2b6" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 45411 } } peers { permanent_uuid: "f542f3f36be44a2d9d93f9dd27051b46" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 33239 } }
I20260811 17:19:32.948596 17609 ts_tablet_manager.cc:1428] T 974297131f7f4fde92b33980ca2812af P 71aef0a85bfe4be5a6113d9918f1c2b6: Time spent starting tablet: real 0.007s user 0.001s sys 0.006s
I20260811 17:19:32.948596 17614 ts_tablet_manager.cc:1428] T 974297131f7f4fde92b33980ca2812af P f542f3f36be44a2d9d93f9dd27051b46: Time spent starting tablet: real 0.006s user 0.001s sys 0.006s
I20260811 17:19:32.949052 17368 heartbeater.cc:503] Master 127.9.157.190:44719 was elected leader, sending a full tablet report...
I20260811 17:19:32.949734 17610 raft_consensus.cc:348] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "281a2fe30fd14636bba35e6d0215694a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39363 } } peers { permanent_uuid: "71aef0a85bfe4be5a6113d9918f1c2b6" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 45411 } } peers { permanent_uuid: "f542f3f36be44a2d9d93f9dd27051b46" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 33239 } }
I20260811 17:19:32.950047 17610 raft_consensus.cc:374] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:32.950102 17610 raft_consensus.cc:729] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 281a2fe30fd14636bba35e6d0215694a, State: Initialized, Role: FOLLOWER
I20260811 17:19:32.950328 17585 heartbeater.cc:503] Master 127.9.157.190:44719 was elected leader, sending a full tablet report...
I20260811 17:19:32.950668 17610 consensus_queue.cc:260] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "281a2fe30fd14636bba35e6d0215694a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39363 } } peers { permanent_uuid: "71aef0a85bfe4be5a6113d9918f1c2b6" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 45411 } } peers { permanent_uuid: "f542f3f36be44a2d9d93f9dd27051b46" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 33239 } }
I20260811 17:19:32.951841 17610 ts_tablet_manager.cc:1428] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
W20260811 17:19:33.114719 17587 tablet.cc:2369] T 974297131f7f4fde92b33980ca2812af P f542f3f36be44a2d9d93f9dd27051b46: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:19:33.114720 17155 tablet.cc:2369] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:19:33.121706 17370 tablet.cc:2369] T 974297131f7f4fde92b33980ca2812af P 71aef0a85bfe4be5a6113d9918f1c2b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:19:33.234762 17624 raft_consensus.cc:482] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:19:33.235075 17624 raft_consensus.cc:504] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "281a2fe30fd14636bba35e6d0215694a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39363 } } peers { permanent_uuid: "71aef0a85bfe4be5a6113d9918f1c2b6" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 45411 } } peers { permanent_uuid: "f542f3f36be44a2d9d93f9dd27051b46" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 33239 } }
I20260811 17:19:33.236611 17624 leader_election.cc:288] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71aef0a85bfe4be5a6113d9918f1c2b6 (127.9.157.130:45411), f542f3f36be44a2d9d93f9dd27051b46 (127.9.157.131:33239)
I20260811 17:19:33.241405 17625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:33.237002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:45411 (local address 127.9.157.129:40443)
0811 17:19:33.237374 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:33.237393 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:33.237530 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:33.237924 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:33.237927 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:33.237949 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:33.238078 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:33.238084 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:33.239031 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:33.239035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:33.240151 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:33.240163 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:33.240938 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:33.240942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:33.241044 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:33.241085 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 17:19:33.241161 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 17:19:33.241243 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":178,"threads_started":1}
I20260811 17:19:33.241525 17607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:33.237081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:40443 (local address 127.9.157.130:45411)
0811 17:19:33.237254 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:33.237259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:33.237645 (+ 386us) server_negotiation.cc:407] Connection header received
0811 17:19:33.237722 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:33.237725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:33.237785 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:33.237890 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:33.238235 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:33.238912 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:33.240294 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:33.240802 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:33.240904 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:33.240951 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:33.241226 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 17:19:33.241342 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 17:19:33.241734 17608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:33.237692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:47161 (local address 127.9.157.131:33239)
0811 17:19:33.237883 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:33.237890 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:33.238149 (+ 259us) server_negotiation.cc:407] Connection header received
0811 17:19:33.238292 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:33.238295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:33.238338 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:33.238397 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:33.238684 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:33.239335 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:33.240537 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:33.241032 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:33.241173 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:33.241221 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:33.241504 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 17:19:33.241598 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 17:19:33.241999 17626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:33.237530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:33239 (local address 127.9.157.129:47161)
0811 17:19:33.237967 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:33.237984 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:33.238177 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:33.238436 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:33.238439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:33.238459 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:33.238575 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:33.238582 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:33.239460 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:33.239462 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:33.240405 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:33.240416 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:33.241217 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:33.241221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:33.241342 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:33.241370 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 17:19:33.241466 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 17:19:33.241723 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":102,"threads_started":1}
I20260811 17:19:33.242355 17290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974297131f7f4fde92b33980ca2812af" candidate_uuid: "281a2fe30fd14636bba35e6d0215694a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71aef0a85bfe4be5a6113d9918f1c2b6" is_pre_election: true
I20260811 17:19:33.242728 17290 raft_consensus.cc:2454] T 974297131f7f4fde92b33980ca2812af P 71aef0a85bfe4be5a6113d9918f1c2b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 281a2fe30fd14636bba35e6d0215694a in term 0.
I20260811 17:19:33.242726 17506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974297131f7f4fde92b33980ca2812af" candidate_uuid: "281a2fe30fd14636bba35e6d0215694a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f542f3f36be44a2d9d93f9dd27051b46" is_pre_election: true
I20260811 17:19:33.243022 17506 raft_consensus.cc:2454] T 974297131f7f4fde92b33980ca2812af P f542f3f36be44a2d9d93f9dd27051b46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 281a2fe30fd14636bba35e6d0215694a in term 0.
I20260811 17:19:33.243316 16963 leader_election.cc:302] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 281a2fe30fd14636bba35e6d0215694a, 71aef0a85bfe4be5a6113d9918f1c2b6; no voters:
I20260811 17:19:33.243630 17624 raft_consensus.cc:2781] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:19:33.243757 17624 raft_consensus.cc:482] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:19:33.243819 17624 raft_consensus.cc:3037] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:33.245086 17624 raft_consensus.cc:504] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "281a2fe30fd14636bba35e6d0215694a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39363 } } peers { permanent_uuid: "71aef0a85bfe4be5a6113d9918f1c2b6" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 45411 } } peers { permanent_uuid: "f542f3f36be44a2d9d93f9dd27051b46" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 33239 } }
I20260811 17:19:33.245496 17624 leader_election.cc:288] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a [CANDIDATE]: Term 1 election: Requested vote from peers 71aef0a85bfe4be5a6113d9918f1c2b6 (127.9.157.130:45411), f542f3f36be44a2d9d93f9dd27051b46 (127.9.157.131:33239)
I20260811 17:19:33.245986 17506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974297131f7f4fde92b33980ca2812af" candidate_uuid: "281a2fe30fd14636bba35e6d0215694a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f542f3f36be44a2d9d93f9dd27051b46"
I20260811 17:19:33.246039 17290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974297131f7f4fde92b33980ca2812af" candidate_uuid: "281a2fe30fd14636bba35e6d0215694a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71aef0a85bfe4be5a6113d9918f1c2b6"
I20260811 17:19:33.246141 17506 raft_consensus.cc:3037] T 974297131f7f4fde92b33980ca2812af P f542f3f36be44a2d9d93f9dd27051b46 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:33.246191 17290 raft_consensus.cc:3037] T 974297131f7f4fde92b33980ca2812af P 71aef0a85bfe4be5a6113d9918f1c2b6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:33.247589 17506 raft_consensus.cc:2454] T 974297131f7f4fde92b33980ca2812af P f542f3f36be44a2d9d93f9dd27051b46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 281a2fe30fd14636bba35e6d0215694a in term 1.
I20260811 17:19:33.247696 17290 raft_consensus.cc:2454] T 974297131f7f4fde92b33980ca2812af P 71aef0a85bfe4be5a6113d9918f1c2b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 281a2fe30fd14636bba35e6d0215694a in term 1.
I20260811 17:19:33.248034 16963 leader_election.cc:302] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 281a2fe30fd14636bba35e6d0215694a, f542f3f36be44a2d9d93f9dd27051b46; no voters:
I20260811 17:19:33.248234 17624 raft_consensus.cc:2781] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:33.248497 17624 raft_consensus.cc:686] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a [term 1 LEADER]: Becoming Leader. State: Replica: 281a2fe30fd14636bba35e6d0215694a, State: Running, Role: LEADER
I20260811 17:19:33.248939 17624 consensus_queue.cc:237] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "281a2fe30fd14636bba35e6d0215694a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39363 } } peers { permanent_uuid: "71aef0a85bfe4be5a6113d9918f1c2b6" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 45411 } } peers { permanent_uuid: "f542f3f36be44a2d9d93f9dd27051b46" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 33239 } }
I20260811 17:19:33.252712 16843 catalog_manager.cc:5495] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a reported cstate change: term changed from 0 to 1, leader changed from <none> to 281a2fe30fd14636bba35e6d0215694a (127.9.157.129). New cstate: current_term: 1 leader_uuid: "281a2fe30fd14636bba35e6d0215694a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "281a2fe30fd14636bba35e6d0215694a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71aef0a85bfe4be5a6113d9918f1c2b6" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 45411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f542f3f36be44a2d9d93f9dd27051b46" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 33239 } health_report { overall_health: UNKNOWN } } }
I20260811 17:19:33.291534 17604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:33.287632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53674 (local address 127.9.157.129:39363)
0811 17:19:33.287823 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:33.287829 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:33.287847 (+ 18us) server_negotiation.cc:407] Connection header received
0811 17:19:33.287970 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:33.287977 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:33.288052 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:33.288150 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:19:33.288431 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:33.289053 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:33.289972 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:33.290185 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:33.290263 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:33.290594 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:19:33.291083 (+ 489us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:19:33.291181 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:33.291330 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 17:19:33.291390 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 17:19:33.298295 17290 raft_consensus.cc:1261] T 974297131f7f4fde92b33980ca2812af P 71aef0a85bfe4be5a6113d9918f1c2b6 [term 1 FOLLOWER]: Refusing update from remote peer 281a2fe30fd14636bba35e6d0215694a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:33.298295 17506 raft_consensus.cc:1261] T 974297131f7f4fde92b33980ca2812af P f542f3f36be44a2d9d93f9dd27051b46 [term 1 FOLLOWER]: Refusing update from remote peer 281a2fe30fd14636bba35e6d0215694a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:33.299274 17624 consensus_queue.cc:1035] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a [LEADER]: Connected to new peer: Peer: permanent_uuid: "71aef0a85bfe4be5a6113d9918f1c2b6" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 45411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:33.299551 17629 consensus_queue.cc:1035] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f542f3f36be44a2d9d93f9dd27051b46" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 33239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:33.310959 17632 mvcc.cc:205] Tried to move back new op lower bound from 7317376095420592128 to 7317376095231008768. Current Snapshot: MvccSnapshot[applied={T|T < 7317376095420592128 or (T in {7317376095420592128})}]
I20260811 17:19:33.319938 16843 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:19:33.320130 16843 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:19:33.322978 16843 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1361af3f3396456787fc3f95a62d3237: Sending DeleteTablet for 3 replicas of tablet 974297131f7f4fde92b33980ca2812af
I20260811 17:19:33.324081 17261 tablet_service.cc:1514] Processing DeleteTablet for tablet 974297131f7f4fde92b33980ca2812af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:33 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:43820
I20260811 17:19:33.324210 17040 tablet_service.cc:1514] Processing DeleteTablet for tablet 974297131f7f4fde92b33980ca2812af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:33 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:53662
I20260811 17:19:33.324273 17471 tablet_service.cc:1514] Processing DeleteTablet for tablet 974297131f7f4fde92b33980ca2812af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:33 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:35540
I20260811 17:19:33.324869 17650 tablet_replica.cc:330] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a: stopping tablet replica
I20260811 17:19:33.325330 17650 raft_consensus.cc:2229] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:19:33.325433 17649 tablet_replica.cc:330] T 974297131f7f4fde92b33980ca2812af P f542f3f36be44a2d9d93f9dd27051b46: stopping tablet replica
I20260811 17:19:33.325824 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 16940
I20260811 17:19:33.325836 17650 raft_consensus.cc:2258] T 974297131f7f4fde92b33980ca2812af P 281a2fe30fd14636bba35e6d0215694a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:33.336805 17649 raft_consensus.cc:2229] T 974297131f7f4fde92b33980ca2812af P f542f3f36be44a2d9d93f9dd27051b46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:19:33.337147 17648 tablet_replica.cc:330] T 974297131f7f4fde92b33980ca2812af P 71aef0a85bfe4be5a6113d9918f1c2b6: stopping tablet replica
I20260811 17:19:33.337333 17649 raft_consensus.cc:2258] T 974297131f7f4fde92b33980ca2812af P f542f3f36be44a2d9d93f9dd27051b46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:33.337581 17648 raft_consensus.cc:2229] T 974297131f7f4fde92b33980ca2812af P 71aef0a85bfe4be5a6113d9918f1c2b6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:19:33.337935 17648 raft_consensus.cc:2258] T 974297131f7f4fde92b33980ca2812af P 71aef0a85bfe4be5a6113d9918f1c2b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:33.339294 17648 ts_tablet_manager.cc:1905] T 974297131f7f4fde92b33980ca2812af P 71aef0a85bfe4be5a6113d9918f1c2b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:33.339299 17649 ts_tablet_manager.cc:1905] T 974297131f7f4fde92b33980ca2812af P f542f3f36be44a2d9d93f9dd27051b46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:33.341889 17648 ts_tablet_manager.cc:1918] T 974297131f7f4fde92b33980ca2812af P 71aef0a85bfe4be5a6113d9918f1c2b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:19:33.342034 17648 log.cc:1196] T 974297131f7f4fde92b33980ca2812af P 71aef0a85bfe4be5a6113d9918f1c2b6: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-1/wal/wals/974297131f7f4fde92b33980ca2812af
I20260811 17:19:33.342281 17649 ts_tablet_manager.cc:1918] T 974297131f7f4fde92b33980ca2812af P f542f3f36be44a2d9d93f9dd27051b46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:19:33.342344 17648 ts_tablet_manager.cc:1939] T 974297131f7f4fde92b33980ca2812af P 71aef0a85bfe4be5a6113d9918f1c2b6: Deleting consensus metadata
I20260811 17:19:33.342396 17649 log.cc:1196] T 974297131f7f4fde92b33980ca2812af P f542f3f36be44a2d9d93f9dd27051b46: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786468743878666-9846-0/minicluster-data/ts-2/wal/wals/974297131f7f4fde92b33980ca2812af
I20260811 17:19:33.342708 17649 ts_tablet_manager.cc:1939] T 974297131f7f4fde92b33980ca2812af P f542f3f36be44a2d9d93f9dd27051b46: Deleting consensus metadata
I20260811 17:19:33.343858 16824 catalog_manager.cc:4841] TS 71aef0a85bfe4be5a6113d9918f1c2b6 (127.9.157.130:45411): tablet 974297131f7f4fde92b33980ca2812af (table test-table [id=b597e913130647ac967b7c57c4624d40]) successfully deleted
I20260811 17:19:33.344146 16824 catalog_manager.cc:4841] TS f542f3f36be44a2d9d93f9dd27051b46 (127.9.157.131:33239): tablet 974297131f7f4fde92b33980ca2812af (table test-table [id=b597e913130647ac967b7c57c4624d40]) successfully deleted
W20260811 17:19:33.344969 16825 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.157.129:39363 (error 108)
I20260811 17:19:33.345148 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 17164
I20260811 17:19:33.360908 17605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:33.360540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:39363 (local address 127.0.0.1:53678)
0811 17:19:33.360710 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:33.360800 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.129:39363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260811 17:19:33.361129 16825 catalog_manager.cc:4584] TS 281a2fe30fd14636bba35e6d0215694a (127.9.157.129:39363): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 974297131f7f4fde92b33980ca2812af: Network error: Client connection negotiation failed: client connection to 127.9.157.129:39363: connect: Connection refused (error 111)
I20260811 17:19:33.363402 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 17375
I20260811 17:19:33.379930 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 16795
2026-08-11T17:19:33Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4359 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T17:19:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:19:33Z Disabled control of system clock
I20260811 17:19:33.427881 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:41981
--webserver_interface=127.9.157.190
--webserver_port=0
--builtin_ntp_servers=127.9.157.148:44307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:41981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 17:19:33.547466 17660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:33.547837 17660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:33.547879 17660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:33.552099 17660 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:19:33.552188 17660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:33.552204 17660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:33.552223 17660 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:19:33.552240 17660 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:19:33.557410 17660 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:44307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:41981
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:41981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:33.558729 17660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:33.560606 17660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:19:33.570533 17660 server_base.cc:1034] running on GCE node
W20260811 17:19:33.570436 17665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:33.570437 17668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:33.570436 17666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:33.571480 17660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:33.572789 17660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:33.574009 17660 hybrid_clock.cc:648] HybridClock initialized: now 1786468773573976 us; error 52 us; skew 500 ppm
I20260811 17:19:33.576285 17660 webserver.cc:460] Webserver started at http://127.9.157.190:46723/ using document root <none> and password file <none>
I20260811 17:19:33.576930 17660 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:33.577028 17660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:33.577281 17660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:33.579490 17660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "cf40dd54c52a4a80962285a390b85cf6"
format_stamp: "Formatted at 2026-08-11 17:19:33 on dist-test-slave-mjjr"
server_key: "87ef1316fa350adb0deb80b4b4dce08c"
server_key_iv: "39c188a540a7163f84d6c67244cf3910"
server_key_version: "encryptionkey@0"
I20260811 17:19:33.580173 17660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "cf40dd54c52a4a80962285a390b85cf6"
format_stamp: "Formatted at 2026-08-11 17:19:33 on dist-test-slave-mjjr"
server_key: "87ef1316fa350adb0deb80b4b4dce08c"
server_key_iv: "39c188a540a7163f84d6c67244cf3910"
server_key_version: "encryptionkey@0"
I20260811 17:19:33.584195 17660 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:19:33.586905 17676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:33.588389 17660 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:33.588554 17660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "cf40dd54c52a4a80962285a390b85cf6"
format_stamp: "Formatted at 2026-08-11 17:19:33 on dist-test-slave-mjjr"
server_key: "87ef1316fa350adb0deb80b4b4dce08c"
server_key_iv: "39c188a540a7163f84d6c67244cf3910"
server_key_version: "encryptionkey@0"
I20260811 17:19:33.588692 17660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:33.593904 17660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:33.594632 17660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:33.594848 17660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:33.602499 17660 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:41981
I20260811 17:19:33.602504 17764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:41981 every 8 connection(s)
I20260811 17:19:33.603603 17660 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:19:33.604750 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 17660
I20260811 17:19:33.604883 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:19:33.605229 9846 external_mini_cluster.cc:1428] Setting key adc5393cd01f20f127c1aa9e9ef6caa6
I20260811 17:19:33.607743 17768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:33.612543 17774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:33.607023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46982 (local address 127.9.157.190:41981)
0811 17:19:33.608435 (+ 1412us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:33.608453 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:33.608495 (+ 42us) server_negotiation.cc:407] Connection header received
0811 17:19:33.609236 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:33.609255 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:33.609644 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:33.610082 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:33.611053 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:33.611058 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:33.611071 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:19:33.611084 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:33.611146 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:19:33.611165 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:33.611299 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:33.611424 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 17:19:33.611737 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":991,"thread_start_us":835,"threads_started":1}
I20260811 17:19:33.614693 17768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf40dd54c52a4a80962285a390b85cf6: Bootstrap starting.
I20260811 17:19:33.617239 17768 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf40dd54c52a4a80962285a390b85cf6: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:33.618039 17768 log.cc:824] T 00000000000000000000000000000000 P cf40dd54c52a4a80962285a390b85cf6: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:33.620612 17768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf40dd54c52a4a80962285a390b85cf6: No bootstrap required, opened a new log
I20260811 17:19:33.624454 17768 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf40dd54c52a4a80962285a390b85cf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf40dd54c52a4a80962285a390b85cf6" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41981 } }
I20260811 17:19:33.624707 17768 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf40dd54c52a4a80962285a390b85cf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:33.624754 17768 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf40dd54c52a4a80962285a390b85cf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf40dd54c52a4a80962285a390b85cf6, State: Initialized, Role: FOLLOWER
I20260811 17:19:33.625252 17768 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf40dd54c52a4a80962285a390b85cf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf40dd54c52a4a80962285a390b85cf6" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41981 } }
I20260811 17:19:33.625435 17768 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf40dd54c52a4a80962285a390b85cf6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:19:33.625510 17768 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf40dd54c52a4a80962285a390b85cf6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:19:33.625618 17768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf40dd54c52a4a80962285a390b85cf6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:33.626732 17768 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf40dd54c52a4a80962285a390b85cf6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf40dd54c52a4a80962285a390b85cf6" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41981 } }
I20260811 17:19:33.627065 17768 leader_election.cc:302] T 00000000000000000000000000000000 P cf40dd54c52a4a80962285a390b85cf6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf40dd54c52a4a80962285a390b85cf6; no voters:
I20260811 17:19:33.627395 17768 leader_election.cc:288] T 00000000000000000000000000000000 P cf40dd54c52a4a80962285a390b85cf6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:19:33.627526 17776 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf40dd54c52a4a80962285a390b85cf6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:33.627847 17776 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf40dd54c52a4a80962285a390b85cf6 [term 1 LEADER]: Becoming Leader. State: Replica: cf40dd54c52a4a80962285a390b85cf6, State: Running, Role: LEADER
I20260811 17:19:33.628186 17776 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf40dd54c52a4a80962285a390b85cf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf40dd54c52a4a80962285a390b85cf6" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41981 } }
I20260811 17:19:33.628801 17768 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf40dd54c52a4a80962285a390b85cf6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:19:33.630180 17780 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf40dd54c52a4a80962285a390b85cf6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf40dd54c52a4a80962285a390b85cf6. Latest consensus state: current_term: 1 leader_uuid: "cf40dd54c52a4a80962285a390b85cf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf40dd54c52a4a80962285a390b85cf6" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41981 } } }
I20260811 17:19:33.630349 17780 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf40dd54c52a4a80962285a390b85cf6 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:33.630177 17779 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf40dd54c52a4a80962285a390b85cf6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf40dd54c52a4a80962285a390b85cf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf40dd54c52a4a80962285a390b85cf6" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41981 } } }
I20260811 17:19:33.630666 17779 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf40dd54c52a4a80962285a390b85cf6 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:33.631361 17789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:19:33.634414 17789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:19:33.640309 17789 catalog_manager.cc:1347] Generated new cluster ID: 06cc67e57cda442193d37048711261ac
I20260811 17:19:33.640414 17789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:19:33.670575 17789 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:19:33.670954 17789 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:19:33.677716 17789 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf40dd54c52a4a80962285a390b85cf6: Generated new TSK 0
I20260811 17:19:33.678514 17789 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:19:33.681553 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:0
--local_ip_for_outbound_sockets=127.9.157.129
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:41981
--builtin_ntp_servers=127.9.157.148:44307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 17:19:33.803574 17812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:33.803937 17812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:33.803982 17812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:33.808215 17812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:33.808295 17812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:33.808396 17812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:19:33.813565 17812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:44307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:41981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:33.815197 17812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:33.817059 17812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:33.828068 17819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:33.828110 17820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:33.828310 17812 server_base.cc:1034] running on GCE node
W20260811 17:19:33.828382 17822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:33.829041 17812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:33.829972 17812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:33.831202 17812 hybrid_clock.cc:648] HybridClock initialized: now 1786468773831174 us; error 44 us; skew 500 ppm
I20260811 17:19:33.833428 17812 webserver.cc:460] Webserver started at http://127.9.157.129:34127/ using document root <none> and password file <none>
I20260811 17:19:33.834097 17812 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:33.834201 17812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:33.834455 17812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:33.836884 17812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-0/data/instance:
uuid: "383beaca2dc8414cb0dfbb6440c831b0"
format_stamp: "Formatted at 2026-08-11 17:19:33 on dist-test-slave-mjjr"
server_key: "7ddd515ccb50336db595c6632853ad49"
server_key_iv: "72adeb351a856117165ee9dbcfb4c953"
server_key_version: "encryptionkey@0"
I20260811 17:19:33.837518 17812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance:
uuid: "383beaca2dc8414cb0dfbb6440c831b0"
format_stamp: "Formatted at 2026-08-11 17:19:33 on dist-test-slave-mjjr"
server_key: "7ddd515ccb50336db595c6632853ad49"
server_key_iv: "72adeb351a856117165ee9dbcfb4c953"
server_key_version: "encryptionkey@0"
I20260811 17:19:33.841867 17812 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:19:33.844666 17837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:33.846055 17812 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:33.846232 17812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "383beaca2dc8414cb0dfbb6440c831b0"
format_stamp: "Formatted at 2026-08-11 17:19:33 on dist-test-slave-mjjr"
server_key: "7ddd515ccb50336db595c6632853ad49"
server_key_iv: "72adeb351a856117165ee9dbcfb4c953"
server_key_version: "encryptionkey@0"
I20260811 17:19:33.846374 17812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:33.851449 17812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:33.852200 17812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:33.852411 17812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:33.853044 17812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:33.854054 17812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:33.854130 17812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:33.854202 17812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:33.854246 17812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:33.863615 17812 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:41561
I20260811 17:19:33.863655 18001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:41561 every 8 connection(s)
I20260811 17:19:33.864903 17812 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:19:33.869926 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 17812
I20260811 17:19:33.870095 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance
I20260811 17:19:33.870460 9846 external_mini_cluster.cc:1428] Setting key 57f77b76e17a19479fbfec4902798763
I20260811 17:19:33.873883 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:0
--local_ip_for_outbound_sockets=127.9.157.130
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:41981
--builtin_ntp_servers=127.9.157.148:44307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 17:19:33.874886 17774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:33.867589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:43897 (local address 127.9.157.190:41981)
0811 17:19:33.867788 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:33.867793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:33.869435 (+ 1642us) server_negotiation.cc:407] Connection header received
0811 17:19:33.872414 (+ 2979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:33.872421 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:33.872471 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:33.872551 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:33.874088 (+ 1537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:33.874093 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:33.874096 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:19:33.874149 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:33.874177 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:19:33.874190 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:33.874345 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:33.874675 (+ 330us) server_negotiation.cc:299] Negotiation successful
0811 17:19:33.874732 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 17:19:33.876381 18006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:33.868203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:41981 (local address 127.9.157.129:43897)
0811 17:19:33.869208 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:33.869275 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:33.872107 (+ 2832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:33.872766 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:33.872778 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:33.872922 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:33.873664 (+ 742us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:19:33.873668 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:33.873732 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:19:33.873844 (+ 112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:33.874425 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:33.874437 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:33.874444 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 17:19:33.874776 (+ 332us) client_negotiation.cc:240] Negotiation successful
0811 17:19:33.875255 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":177,"threads_started":1}
I20260811 17:19:33.877919 18002 heartbeater.cc:347] Connected to a master server at 127.9.157.190:41981
I20260811 17:19:33.878223 18002 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:33.878809 18002 heartbeater.cc:511] Master 127.9.157.190:41981 requested a full tablet report, sending...
I20260811 17:19:33.880681 17707 ts_manager.cc:194] Registered new tserver with Master: 383beaca2dc8414cb0dfbb6440c831b0 (127.9.157.129:41561)
W20260811 17:19:34.000180 18007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:34.000548 18007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:34.000629 18007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:34.004902 18007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:34.005014 18007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:34.005121 18007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:19:34.010265 18007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:44307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:41981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:34.011548 18007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:34.013108 18007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:34.023860 18012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:34.023886 18018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:34.024102 18007 server_base.cc:1034] running on GCE node
W20260811 17:19:34.023860 18022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:34.024696 18007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:34.025444 18007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:34.026664 18007 hybrid_clock.cc:648] HybridClock initialized: now 1786468774026646 us; error 33 us; skew 500 ppm
I20260811 17:19:34.028934 18007 webserver.cc:460] Webserver started at http://127.9.157.130:36871/ using document root <none> and password file <none>
I20260811 17:19:34.029695 18007 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:34.029803 18007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:34.030057 18007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:34.032657 18007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-1/data/instance:
uuid: "022b28b010c649559e04a50665ecde82"
format_stamp: "Formatted at 2026-08-11 17:19:34 on dist-test-slave-mjjr"
server_key: "bec34022b668ca39d86b2e14dbd552dd"
server_key_iv: "6a17e9922c566fc934048fc15cd1362e"
server_key_version: "encryptionkey@0"
I20260811 17:19:34.033445 18007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance:
uuid: "022b28b010c649559e04a50665ecde82"
format_stamp: "Formatted at 2026-08-11 17:19:34 on dist-test-slave-mjjr"
server_key: "bec34022b668ca39d86b2e14dbd552dd"
server_key_iv: "6a17e9922c566fc934048fc15cd1362e"
server_key_version: "encryptionkey@0"
I20260811 17:19:34.037861 18007 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:19:34.040723 18029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:34.042171 18007 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:34.042364 18007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "022b28b010c649559e04a50665ecde82"
format_stamp: "Formatted at 2026-08-11 17:19:34 on dist-test-slave-mjjr"
server_key: "bec34022b668ca39d86b2e14dbd552dd"
server_key_iv: "6a17e9922c566fc934048fc15cd1362e"
server_key_version: "encryptionkey@0"
I20260811 17:19:34.042510 18007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:34.047781 18007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:34.048668 18007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:34.048898 18007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:34.049714 18007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:34.051149 18007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:34.051230 18007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:34.051291 18007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:34.051333 18007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:34.064847 18007 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:38995
I20260811 17:19:34.064868 18207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:38995 every 8 connection(s)
I20260811 17:19:34.065898 18007 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:19:34.072458 17774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:34.069467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:55717 (local address 127.9.157.190:41981)
0811 17:19:34.069602 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:34.069606 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:34.069621 (+ 15us) server_negotiation.cc:407] Connection header received
0811 17:19:34.070708 (+ 1087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:34.070711 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:34.070751 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:34.070839 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:34.072035 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:34.072038 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:34.072040 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:19:34.072053 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:34.072072 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:19:34.072080 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:34.072161 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:34.072298 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 17:19:34.072340 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 17:19:34.073417 18210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:34.068345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:41981 (local address 127.9.157.130:55717)
0811 17:19:34.069075 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:34.069120 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:34.070481 (+ 1361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:34.071026 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:34.071033 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:34.071129 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:34.071766 (+ 637us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:19:34.071768 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:34.071811 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:19:34.071887 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:34.072160 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:34.072167 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:34.072172 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 17:19:34.072327 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 17:19:34.072571 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":112,"threads_started":1}
I20260811 17:19:34.073959 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 18007
I20260811 17:19:34.074098 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance
I20260811 17:19:34.074427 9846 external_mini_cluster.cc:1428] Setting key 94e96a089c42e013f241043ef1ff78f7
I20260811 17:19:34.074756 18208 heartbeater.cc:347] Connected to a master server at 127.9.157.190:41981
I20260811 17:19:34.075033 18208 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:34.075502 18208 heartbeater.cc:511] Master 127.9.157.190:41981 requested a full tablet report, sending...
I20260811 17:19:34.076751 17707 ts_manager.cc:194] Registered new tserver with Master: 022b28b010c649559e04a50665ecde82 (127.9.157.130:38995)
I20260811 17:19:34.076789 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:0
--local_ip_for_outbound_sockets=127.9.157.131
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:41981
--builtin_ntp_servers=127.9.157.148:44307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 17:19:34.199538 18211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:34.199931 18211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:34.200017 18211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:34.204309 18211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:34.204419 18211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:34.204524 18211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:19:34.209618 18211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:44307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:41981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:34.210932 18211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:34.212538 18211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:34.223621 18222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:34.223558 18220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:34.223575 18211 server_base.cc:1034] running on GCE node
W20260811 17:19:34.223558 18225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:34.224391 18211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:34.225199 18211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:34.226416 18211 hybrid_clock.cc:648] HybridClock initialized: now 1786468774226396 us; error 34 us; skew 500 ppm
I20260811 17:19:34.228632 18211 webserver.cc:460] Webserver started at http://127.9.157.131:45869/ using document root <none> and password file <none>
I20260811 17:19:34.229403 18211 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:34.229509 18211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:34.229784 18211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:34.232337 18211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-2/data/instance:
uuid: "e246856c479d4f469526df84eb66d66c"
format_stamp: "Formatted at 2026-08-11 17:19:34 on dist-test-slave-mjjr"
server_key: "d014e534013052e65600ef90ebb50eb3"
server_key_iv: "1ace7d8fa8bde2a40a50f710479cb0a4"
server_key_version: "encryptionkey@0"
I20260811 17:19:34.233024 18211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance:
uuid: "e246856c479d4f469526df84eb66d66c"
format_stamp: "Formatted at 2026-08-11 17:19:34 on dist-test-slave-mjjr"
server_key: "d014e534013052e65600ef90ebb50eb3"
server_key_iv: "1ace7d8fa8bde2a40a50f710479cb0a4"
server_key_version: "encryptionkey@0"
I20260811 17:19:34.237794 18211 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:19:34.241037 18235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:34.242532 18211 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 17:19:34.242709 18211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "e246856c479d4f469526df84eb66d66c"
format_stamp: "Formatted at 2026-08-11 17:19:34 on dist-test-slave-mjjr"
server_key: "d014e534013052e65600ef90ebb50eb3"
server_key_iv: "1ace7d8fa8bde2a40a50f710479cb0a4"
server_key_version: "encryptionkey@0"
I20260811 17:19:34.242861 18211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:34.248250 18211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:34.248941 18211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:34.249142 18211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:34.249745 18211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:34.250826 18211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:34.250897 18211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:34.250974 18211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:34.251029 18211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:34.260749 18211 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:45171
I20260811 17:19:34.260783 18426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:45171 every 8 connection(s)
I20260811 17:19:34.261762 18211 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:19:34.264792 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 18211
I20260811 17:19:34.264962 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance
I20260811 17:19:34.265264 9846 external_mini_cluster.cc:1428] Setting key fa3ecf1e2b1a78cc7c2ac5bac19f2499
I20260811 17:19:34.269218 17774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:34.263938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:33371 (local address 127.9.157.190:41981)
0811 17:19:34.264113 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:34.264119 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:34.265546 (+ 1427us) server_negotiation.cc:407] Connection header received
0811 17:19:34.267273 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:34.267277 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:34.267324 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:34.267395 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:34.268701 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:34.268705 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:34.268708 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:19:34.268721 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:34.268744 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:19:34.268764 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:34.268863 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:34.269023 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 17:19:34.269072 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 17:19:34.270165 18429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:34.264476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:41981 (local address 127.9.157.131:33371)
0811 17:19:34.265337 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:34.265399 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:34.267029 (+ 1630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:34.267576 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:34.267584 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:34.267745 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:34.268414 (+ 669us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:19:34.268415 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:34.268460 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:19:34.268538 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:34.268851 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:34.268861 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:34.268868 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 17:19:34.269075 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 17:19:34.269357 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":187,"threads_started":1}
I20260811 17:19:34.271375 18427 heartbeater.cc:347] Connected to a master server at 127.9.157.190:41981
I20260811 17:19:34.271610 18427 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:34.272083 18427 heartbeater.cc:511] Master 127.9.157.190:41981 requested a full tablet report, sending...
I20260811 17:19:34.273182 17707 ts_manager.cc:194] Registered new tserver with Master: e246856c479d4f469526df84eb66d66c (127.9.157.131:45171)
I20260811 17:19:34.280098 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:19:34.284135 17774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:34.282358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46984 (local address 127.9.157.190:41981)
0811 17:19:34.282559 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:34.282564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:34.282622 (+ 58us) server_negotiation.cc:407] Connection header received
0811 17:19:34.282761 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:34.282764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:34.282802 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:34.282877 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:34.283679 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:34.283683 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:34.283686 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:19:34.283702 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:34.283725 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:19:34.283734 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:34.283828 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:34.283891 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 17:19:34.283935 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 17:19:34.287446 17707 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46984:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:19:34.290697 17707 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:19:34.304723 18450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:34.301254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:41561 (local address 127.0.0.1:51940)
0811 17:19:34.302297 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:34.302362 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:34.302489 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:34.303029 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:34.303036 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:34.303119 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:34.303849 (+ 730us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:19:34.303851 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:34.303875 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:19:34.303958 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:34.304314 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:34.304328 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:34.304335 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 17:19:34.304445 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 17:19:34.304507 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":833,"thread_start_us":114,"threads_started":1}
I20260811 17:19:34.305081 18456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:34.302080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51940 (local address 127.9.157.129:41561)
0811 17:19:34.302523 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:34.302535 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:34.302563 (+ 28us) server_negotiation.cc:407] Connection header received
0811 17:19:34.302625 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:34.302632 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:34.302868 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:34.303049 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:34.304093 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:34.304100 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:34.304112 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:19:34.304122 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:34.304167 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:19:34.304198 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:34.304763 (+ 565us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:34.304821 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 17:19:34.304915 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":120,"threads_started":1}
I20260811 17:19:34.305464 18447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:34.301494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:45171 (local address 127.0.0.1:59826)
0811 17:19:34.302649 (+ 1155us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:34.302666 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:34.302820 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:34.304100 (+ 1280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:34.304104 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:34.304120 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:34.304658 (+ 538us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:19:34.304659 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:34.304671 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:19:34.304708 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:34.305196 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:34.305201 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:34.305205 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 17:19:34.305288 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 17:19:34.305329 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1077,"thread_start_us":129,"threads_started":1}
I20260811 17:19:34.305701 18454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:34.301714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59826 (local address 127.9.157.131:45171)
0811 17:19:34.302235 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:34.302250 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:34.302764 (+ 514us) server_negotiation.cc:407] Connection header received
0811 17:19:34.302973 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:34.302982 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:34.303179 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:34.303358 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:34.304881 (+ 1523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:34.304890 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:34.304903 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:19:34.304919 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:34.304969 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:19:34.305014 (+ 45us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:34.305106 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:34.305321 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 17:19:34.305399 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":170,"threads_started":1}
I20260811 17:19:34.306527 18455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:34.301266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:38995 (local address 127.0.0.1:59770)
0811 17:19:34.302711 (+ 1445us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:34.302726 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:34.302812 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:34.304102 (+ 1290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:34.304105 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:34.304123 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:34.304737 (+ 614us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:19:34.304739 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:34.304751 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:19:34.304781 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:34.306233 (+ 1452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:34.306236 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:34.306240 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 17:19:34.306323 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 17:19:34.306362 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1370,"mutex_wait_us":743,"thread_start_us":109,"threads_started":1}
I20260811 17:19:34.307063 18451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:34.301719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59770 (local address 127.9.157.130:38995)
0811 17:19:34.302492 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:34.302506 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:34.302930 (+ 424us) server_negotiation.cc:407] Connection header received
0811 17:19:34.302998 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:34.303004 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:34.303182 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:34.303331 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:34.305931 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:34.305940 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:34.305952 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:19:34.305969 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:34.306020 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:19:34.306063 (+ 43us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:34.306181 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:34.306850 (+ 669us) server_negotiation.cc:299] Negotiation successful
0811 17:19:34.306918 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":113,"threads_started":1}
I20260811 17:19:34.308540 17901 tablet_service.cc:1467] Processing CreateTablet for tablet 67978fc439014de2a0c3087842a314eb (DEFAULT_TABLE table=test-table [id=786a9bedac3f40269bff250978dcfb2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:34.308542 18314 tablet_service.cc:1467] Processing CreateTablet for tablet 67978fc439014de2a0c3087842a314eb (DEFAULT_TABLE table=test-table [id=786a9bedac3f40269bff250978dcfb2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:34.309798 17901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67978fc439014de2a0c3087842a314eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:34.309798 18314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67978fc439014de2a0c3087842a314eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:34.309768 18103 tablet_service.cc:1467] Processing CreateTablet for tablet 67978fc439014de2a0c3087842a314eb (DEFAULT_TABLE table=test-table [id=786a9bedac3f40269bff250978dcfb2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:34.310642 18103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67978fc439014de2a0c3087842a314eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:34.318871 18459 tablet_bootstrap.cc:492] T 67978fc439014de2a0c3087842a314eb P 022b28b010c649559e04a50665ecde82: Bootstrap starting.
I20260811 17:19:34.319209 18457 tablet_bootstrap.cc:492] T 67978fc439014de2a0c3087842a314eb P 383beaca2dc8414cb0dfbb6440c831b0: Bootstrap starting.
I20260811 17:19:34.319254 18458 tablet_bootstrap.cc:492] T 67978fc439014de2a0c3087842a314eb P e246856c479d4f469526df84eb66d66c: Bootstrap starting.
I20260811 17:19:34.321331 18457 tablet_bootstrap.cc:654] T 67978fc439014de2a0c3087842a314eb P 383beaca2dc8414cb0dfbb6440c831b0: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:34.321664 18458 tablet_bootstrap.cc:654] T 67978fc439014de2a0c3087842a314eb P e246856c479d4f469526df84eb66d66c: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:34.321664 18459 tablet_bootstrap.cc:654] T 67978fc439014de2a0c3087842a314eb P 022b28b010c649559e04a50665ecde82: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:34.322203 18457 log.cc:824] T 67978fc439014de2a0c3087842a314eb P 383beaca2dc8414cb0dfbb6440c831b0: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:34.322602 18458 log.cc:824] T 67978fc439014de2a0c3087842a314eb P e246856c479d4f469526df84eb66d66c: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:34.322602 18459 log.cc:824] T 67978fc439014de2a0c3087842a314eb P 022b28b010c649559e04a50665ecde82: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:34.324447 18457 tablet_bootstrap.cc:492] T 67978fc439014de2a0c3087842a314eb P 383beaca2dc8414cb0dfbb6440c831b0: No bootstrap required, opened a new log
I20260811 17:19:34.324695 18457 ts_tablet_manager.cc:1397] T 67978fc439014de2a0c3087842a314eb P 383beaca2dc8414cb0dfbb6440c831b0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.002s
I20260811 17:19:34.324821 18458 tablet_bootstrap.cc:492] T 67978fc439014de2a0c3087842a314eb P e246856c479d4f469526df84eb66d66c: No bootstrap required, opened a new log
I20260811 17:19:34.325031 18458 ts_tablet_manager.cc:1397] T 67978fc439014de2a0c3087842a314eb P e246856c479d4f469526df84eb66d66c: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 17:19:34.325249 18459 tablet_bootstrap.cc:492] T 67978fc439014de2a0c3087842a314eb P 022b28b010c649559e04a50665ecde82: No bootstrap required, opened a new log
I20260811 17:19:34.325446 18459 ts_tablet_manager.cc:1397] T 67978fc439014de2a0c3087842a314eb P 022b28b010c649559e04a50665ecde82: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 17:19:34.328820 18458 raft_consensus.cc:348] T 67978fc439014de2a0c3087842a314eb P e246856c479d4f469526df84eb66d66c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022b28b010c649559e04a50665ecde82" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 38995 } } peers { permanent_uuid: "383beaca2dc8414cb0dfbb6440c831b0" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 41561 } } peers { permanent_uuid: "e246856c479d4f469526df84eb66d66c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 45171 } }
I20260811 17:19:34.329202 18458 raft_consensus.cc:374] T 67978fc439014de2a0c3087842a314eb P e246856c479d4f469526df84eb66d66c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:34.329313 18458 raft_consensus.cc:729] T 67978fc439014de2a0c3087842a314eb P e246856c479d4f469526df84eb66d66c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e246856c479d4f469526df84eb66d66c, State: Initialized, Role: FOLLOWER
I20260811 17:19:34.329733 18457 raft_consensus.cc:348] T 67978fc439014de2a0c3087842a314eb P 383beaca2dc8414cb0dfbb6440c831b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022b28b010c649559e04a50665ecde82" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 38995 } } peers { permanent_uuid: "383beaca2dc8414cb0dfbb6440c831b0" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 41561 } } peers { permanent_uuid: "e246856c479d4f469526df84eb66d66c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 45171 } }
I20260811 17:19:34.329990 18457 raft_consensus.cc:374] T 67978fc439014de2a0c3087842a314eb P 383beaca2dc8414cb0dfbb6440c831b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:34.329900 18458 consensus_queue.cc:260] T 67978fc439014de2a0c3087842a314eb P e246856c479d4f469526df84eb66d66c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022b28b010c649559e04a50665ecde82" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 38995 } } peers { permanent_uuid: "383beaca2dc8414cb0dfbb6440c831b0" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 41561 } } peers { permanent_uuid: "e246856c479d4f469526df84eb66d66c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 45171 } }
I20260811 17:19:34.330040 18457 raft_consensus.cc:729] T 67978fc439014de2a0c3087842a314eb P 383beaca2dc8414cb0dfbb6440c831b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 383beaca2dc8414cb0dfbb6440c831b0, State: Initialized, Role: FOLLOWER
I20260811 17:19:34.330096 18459 raft_consensus.cc:348] T 67978fc439014de2a0c3087842a314eb P 022b28b010c649559e04a50665ecde82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022b28b010c649559e04a50665ecde82" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 38995 } } peers { permanent_uuid: "383beaca2dc8414cb0dfbb6440c831b0" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 41561 } } peers { permanent_uuid: "e246856c479d4f469526df84eb66d66c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 45171 } }
I20260811 17:19:34.330335 18459 raft_consensus.cc:374] T 67978fc439014de2a0c3087842a314eb P 022b28b010c649559e04a50665ecde82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:34.330404 18459 raft_consensus.cc:729] T 67978fc439014de2a0c3087842a314eb P 022b28b010c649559e04a50665ecde82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 022b28b010c649559e04a50665ecde82, State: Initialized, Role: FOLLOWER
I20260811 17:19:34.330513 18457 consensus_queue.cc:260] T 67978fc439014de2a0c3087842a314eb P 383beaca2dc8414cb0dfbb6440c831b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022b28b010c649559e04a50665ecde82" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 38995 } } peers { permanent_uuid: "383beaca2dc8414cb0dfbb6440c831b0" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 41561 } } peers { permanent_uuid: "e246856c479d4f469526df84eb66d66c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 45171 } }
I20260811 17:19:34.330888 18427 heartbeater.cc:503] Master 127.9.157.190:41981 was elected leader, sending a full tablet report...
I20260811 17:19:34.330989 18459 consensus_queue.cc:260] T 67978fc439014de2a0c3087842a314eb P 022b28b010c649559e04a50665ecde82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022b28b010c649559e04a50665ecde82" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 38995 } } peers { permanent_uuid: "383beaca2dc8414cb0dfbb6440c831b0" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 41561 } } peers { permanent_uuid: "e246856c479d4f469526df84eb66d66c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 45171 } }
I20260811 17:19:34.331207 18458 ts_tablet_manager.cc:1428] T 67978fc439014de2a0c3087842a314eb P e246856c479d4f469526df84eb66d66c: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260811 17:19:34.331663 18457 ts_tablet_manager.cc:1428] T 67978fc439014de2a0c3087842a314eb P 383beaca2dc8414cb0dfbb6440c831b0: Time spent starting tablet: real 0.007s user 0.004s sys 0.002s
I20260811 17:19:34.332027 18002 heartbeater.cc:503] Master 127.9.157.190:41981 was elected leader, sending a full tablet report...
I20260811 17:19:34.333115 18459 ts_tablet_manager.cc:1428] T 67978fc439014de2a0c3087842a314eb P 022b28b010c649559e04a50665ecde82: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260811 17:19:34.333446 18208 heartbeater.cc:503] Master 127.9.157.190:41981 was elected leader, sending a full tablet report...
W20260811 17:19:34.367909 18003 tablet.cc:2369] T 67978fc439014de2a0c3087842a314eb P 383beaca2dc8414cb0dfbb6440c831b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:19:34.419209 18468 raft_consensus.cc:482] T 67978fc439014de2a0c3087842a314eb P 383beaca2dc8414cb0dfbb6440c831b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:19:34.419520 18468 raft_consensus.cc:504] T 67978fc439014de2a0c3087842a314eb P 383beaca2dc8414cb0dfbb6440c831b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022b28b010c649559e04a50665ecde82" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 38995 } } peers { permanent_uuid: "383beaca2dc8414cb0dfbb6440c831b0" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 41561 } } peers { permanent_uuid: "e246856c479d4f469526df84eb66d66c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 45171 } }
I20260811 17:19:34.420989 18468 leader_election.cc:288] T 67978fc439014de2a0c3087842a314eb P 383beaca2dc8414cb0dfbb6440c831b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 022b28b010c649559e04a50665ecde82 (127.9.157.130:38995), e246856c479d4f469526df84eb66d66c (127.9.157.131:45171)
I20260811 17:19:34.423395 18454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:34.421455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:44241 (local address 127.9.157.131:45171)
0811 17:19:34.421691 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:34.421698 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:34.421722 (+ 24us) server_negotiation.cc:407] Connection header received
0811 17:19:34.421893 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:34.421897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:34.421942 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:34.422039 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:34.422880 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:34.422884 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:34.422887 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:19:34.422903 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:34.422925 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:19:34.422938 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:34.423041 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:34.423193 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 17:19:34.423244 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260811 17:19:34.423440 18472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:34.421290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:45171 (local address 127.9.157.129:44241)
0811 17:19:34.421641 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:34.421658 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:34.421779 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:34.422066 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:34.422069 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:34.422084 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:34.422645 (+ 561us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:19:34.422650 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:34.422666 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:19:34.422703 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:34.423054 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:34.423058 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:34.423062 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 17:19:34.423177 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 17:19:34.423228 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":99,"threads_started":1}
I20260811 17:19:34.423467 18451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:34.421448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:58257 (local address 127.9.157.130:38995)
0811 17:19:34.421659 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:34.421664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:34.421840 (+ 176us) server_negotiation.cc:407] Connection header received
0811 17:19:34.421981 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:34.421993 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:34.422028 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:34.422118 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:34.423064 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:34.423068 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:34.423070 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:19:34.423084 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:34.423111 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:19:34.423124 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:34.423219 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:34.423304 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 17:19:34.423344 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 17:19:34.423575 18471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:34.421318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:38995 (local address 127.9.157.129:58257)
0811 17:19:34.421748 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:34.421760 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:34.421877 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:34.422137 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:34.422139 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:34.422155 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:34.422833 (+ 678us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:19:34.422835 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:34.422846 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:19:34.422874 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:34.423218 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:34.423222 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:34.423225 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 17:19:34.423307 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 17:19:34.423367 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":102,"threads_started":1}
I20260811 17:19:34.424361 18138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67978fc439014de2a0c3087842a314eb" candidate_uuid: "383beaca2dc8414cb0dfbb6440c831b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "022b28b010c649559e04a50665ecde82" is_pre_election: true
I20260811 17:19:34.424353 18357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67978fc439014de2a0c3087842a314eb" candidate_uuid: "383beaca2dc8414cb0dfbb6440c831b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e246856c479d4f469526df84eb66d66c" is_pre_election: true
I20260811 17:19:34.424743 18357 raft_consensus.cc:2454] T 67978fc439014de2a0c3087842a314eb P e246856c479d4f469526df84eb66d66c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 383beaca2dc8414cb0dfbb6440c831b0 in term 0.
I20260811 17:19:34.424758 18138 raft_consensus.cc:2454] T 67978fc439014de2a0c3087842a314eb P 022b28b010c649559e04a50665ecde82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 383beaca2dc8414cb0dfbb6440c831b0 in term 0.
I20260811 17:19:34.425398 17840 leader_election.cc:302] T 67978fc439014de2a0c3087842a314eb P 383beaca2dc8414cb0dfbb6440c831b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 022b28b010c649559e04a50665ecde82, 383beaca2dc8414cb0dfbb6440c831b0, e246856c479d4f469526df84eb66d66c; no voters:
I20260811 17:19:34.425781 18468 raft_consensus.cc:2781] T 67978fc439014de2a0c3087842a314eb P 383beaca2dc8414cb0dfbb6440c831b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:19:34.425875 18468 raft_consensus.cc:482] T 67978fc439014de2a0c3087842a314eb P 383beaca2dc8414cb0dfbb6440c831b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:19:34.425908 18468 raft_consensus.cc:3037] T 67978fc439014de2a0c3087842a314eb P 383beaca2dc8414cb0dfbb6440c831b0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:34.427105 18468 raft_consensus.cc:504] T 67978fc439014de2a0c3087842a314eb P 383beaca2dc8414cb0dfbb6440c831b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022b28b010c649559e04a50665ecde82" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 38995 } } peers { permanent_uuid: "383beaca2dc8414cb0dfbb6440c831b0" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 41561 } } peers { permanent_uuid: "e246856c479d4f469526df84eb66d66c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 45171 } }
I20260811 17:19:34.427525 18468 leader_election.cc:288] T 67978fc439014de2a0c3087842a314eb P 383beaca2dc8414cb0dfbb6440c831b0 [CANDIDATE]: Term 1 election: Requested vote from peers 022b28b010c649559e04a50665ecde82 (127.9.157.130:38995), e246856c479d4f469526df84eb66d66c (127.9.157.131:45171)
I20260811 17:19:34.428063 18138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67978fc439014de2a0c3087842a314eb" candidate_uuid: "383beaca2dc8414cb0dfbb6440c831b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "022b28b010c649559e04a50665ecde82"
I20260811 17:19:34.428157 18357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67978fc439014de2a0c3087842a314eb" candidate_uuid: "383beaca2dc8414cb0dfbb6440c831b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e246856c479d4f469526df84eb66d66c"
I20260811 17:19:34.428252 18138 raft_consensus.cc:3037] T 67978fc439014de2a0c3087842a314eb P 022b28b010c649559e04a50665ecde82 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:34.428323 18357 raft_consensus.cc:3037] T 67978fc439014de2a0c3087842a314eb P e246856c479d4f469526df84eb66d66c [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:34.429668 18357 raft_consensus.cc:2454] T 67978fc439014de2a0c3087842a314eb P e246856c479d4f469526df84eb66d66c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 383beaca2dc8414cb0dfbb6440c831b0 in term 1.
I20260811 17:19:34.429677 18138 raft_consensus.cc:2454] T 67978fc439014de2a0c3087842a314eb P 022b28b010c649559e04a50665ecde82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 383beaca2dc8414cb0dfbb6440c831b0 in term 1.
I20260811 17:19:34.430130 17843 leader_election.cc:302] T 67978fc439014de2a0c3087842a314eb P 383beaca2dc8414cb0dfbb6440c831b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 022b28b010c649559e04a50665ecde82, 383beaca2dc8414cb0dfbb6440c831b0, e246856c479d4f469526df84eb66d66c; no voters:
I20260811 17:19:34.430397 18468 raft_consensus.cc:2781] T 67978fc439014de2a0c3087842a314eb P 383beaca2dc8414cb0dfbb6440c831b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:34.430655 18468 raft_consensus.cc:686] T 67978fc439014de2a0c3087842a314eb P 383beaca2dc8414cb0dfbb6440c831b0 [term 1 LEADER]: Becoming Leader. State: Replica: 383beaca2dc8414cb0dfbb6440c831b0, State: Running, Role: LEADER
I20260811 17:19:34.431115 18468 consensus_queue.cc:237] T 67978fc439014de2a0c3087842a314eb P 383beaca2dc8414cb0dfbb6440c831b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022b28b010c649559e04a50665ecde82" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 38995 } } peers { permanent_uuid: "383beaca2dc8414cb0dfbb6440c831b0" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 41561 } } peers { permanent_uuid: "e246856c479d4f469526df84eb66d66c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 45171 } }
I20260811 17:19:34.434715 17705 catalog_manager.cc:5495] T 67978fc439014de2a0c3087842a314eb P 383beaca2dc8414cb0dfbb6440c831b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 383beaca2dc8414cb0dfbb6440c831b0 (127.9.157.129). New cstate: current_term: 1 leader_uuid: "383beaca2dc8414cb0dfbb6440c831b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "022b28b010c649559e04a50665ecde82" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 38995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "383beaca2dc8414cb0dfbb6440c831b0" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 41561 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e246856c479d4f469526df84eb66d66c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 45171 } health_report { overall_health: UNKNOWN } } }
I20260811 17:19:34.461735 18456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:34.460321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51954 (local address 127.9.157.129:41561)
0811 17:19:34.460459 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:34.460463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:34.460478 (+ 15us) server_negotiation.cc:407] Connection header received
0811 17:19:34.460567 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:34.460569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:34.460633 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:34.460712 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:34.461389 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:34.461392 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:34.461393 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:19:34.461407 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:34.461427 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:19:34.461435 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:34.461516 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:34.461584 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 17:19:34.461621 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 17:19:34.468154 18138 raft_consensus.cc:1261] T 67978fc439014de2a0c3087842a314eb P 022b28b010c649559e04a50665ecde82 [term 1 FOLLOWER]: Refusing update from remote peer 383beaca2dc8414cb0dfbb6440c831b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:34.468155 18357 raft_consensus.cc:1261] T 67978fc439014de2a0c3087842a314eb P e246856c479d4f469526df84eb66d66c [term 1 FOLLOWER]: Refusing update from remote peer 383beaca2dc8414cb0dfbb6440c831b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:34.468993 18473 consensus_queue.cc:1035] T 67978fc439014de2a0c3087842a314eb P 383beaca2dc8414cb0dfbb6440c831b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e246856c479d4f469526df84eb66d66c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:34.469195 18468 consensus_queue.cc:1035] T 67978fc439014de2a0c3087842a314eb P 383beaca2dc8414cb0dfbb6440c831b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "022b28b010c649559e04a50665ecde82" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 38995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:34.476132 18477 mvcc.cc:205] Tried to move back new op lower bound from 7317376100213776384 to 7317376100073275392. Current Snapshot: MvccSnapshot[applied={T|T < 7317376100213776384}]
I20260811 17:19:34.477836 18479 mvcc.cc:205] Tried to move back new op lower bound from 7317376100213776384 to 7317376100073275392. Current Snapshot: MvccSnapshot[applied={T|T < 7317376100213776384}]
I20260811 17:19:34.486542 17705 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:19:34.486757 17705 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:19:34.489768 17705 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cf40dd54c52a4a80962285a390b85cf6: Sending DeleteTablet for 3 replicas of tablet 67978fc439014de2a0c3087842a314eb
I20260811 17:19:34.490836 17901 tablet_service.cc:1514] Processing DeleteTablet for tablet 67978fc439014de2a0c3087842a314eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:34 UTC) from {username='slave'} at 127.0.0.1:51940
I20260811 17:19:34.491001 18314 tablet_service.cc:1514] Processing DeleteTablet for tablet 67978fc439014de2a0c3087842a314eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:34 UTC) from {username='slave'} at 127.0.0.1:59826
I20260811 17:19:34.491477 18103 tablet_service.cc:1514] Processing DeleteTablet for tablet 67978fc439014de2a0c3087842a314eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:34 UTC) from {username='slave'} at 127.0.0.1:59770
I20260811 17:19:34.491937 18492 tablet_replica.cc:330] T 67978fc439014de2a0c3087842a314eb P e246856c479d4f469526df84eb66d66c: stopping tablet replica
I20260811 17:19:34.492108 18493 tablet_replica.cc:330] T 67978fc439014de2a0c3087842a314eb P 022b28b010c649559e04a50665ecde82: stopping tablet replica
I20260811 17:19:34.492419 18492 raft_consensus.cc:2229] T 67978fc439014de2a0c3087842a314eb P e246856c479d4f469526df84eb66d66c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:19:34.492657 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 17812
I20260811 17:19:34.492796 18493 raft_consensus.cc:2229] T 67978fc439014de2a0c3087842a314eb P 022b28b010c649559e04a50665ecde82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:19:34.493156 18493 raft_consensus.cc:2258] T 67978fc439014de2a0c3087842a314eb P 022b28b010c649559e04a50665ecde82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:34.493010 18492 raft_consensus.cc:2258] T 67978fc439014de2a0c3087842a314eb P e246856c479d4f469526df84eb66d66c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:34.497290 18493 ts_tablet_manager.cc:1905] T 67978fc439014de2a0c3087842a314eb P 022b28b010c649559e04a50665ecde82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:34.499212 18492 ts_tablet_manager.cc:1905] T 67978fc439014de2a0c3087842a314eb P e246856c479d4f469526df84eb66d66c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:34.500985 18493 ts_tablet_manager.cc:1918] T 67978fc439014de2a0c3087842a314eb P 022b28b010c649559e04a50665ecde82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:19:34.501157 18493 log.cc:1196] T 67978fc439014de2a0c3087842a314eb P 022b28b010c649559e04a50665ecde82: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-1/wal/wals/67978fc439014de2a0c3087842a314eb
I20260811 17:19:34.501579 18493 ts_tablet_manager.cc:1939] T 67978fc439014de2a0c3087842a314eb P 022b28b010c649559e04a50665ecde82: Deleting consensus metadata
I20260811 17:19:34.502446 18492 ts_tablet_manager.cc:1918] T 67978fc439014de2a0c3087842a314eb P e246856c479d4f469526df84eb66d66c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:19:34.502579 18492 log.cc:1196] T 67978fc439014de2a0c3087842a314eb P e246856c479d4f469526df84eb66d66c: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786468743878666-9846-0/minicluster-data/ts-2/wal/wals/67978fc439014de2a0c3087842a314eb
I20260811 17:19:34.502907 18492 ts_tablet_manager.cc:1939] T 67978fc439014de2a0c3087842a314eb P e246856c479d4f469526df84eb66d66c: Deleting consensus metadata
I20260811 17:19:34.503528 17679 catalog_manager.cc:4841] TS 022b28b010c649559e04a50665ecde82 (127.9.157.130:38995): tablet 67978fc439014de2a0c3087842a314eb (table test-table [id=786a9bedac3f40269bff250978dcfb2e]) successfully deleted
I20260811 17:19:34.504175 17683 catalog_manager.cc:4841] TS e246856c479d4f469526df84eb66d66c (127.9.157.131:45171): tablet 67978fc439014de2a0c3087842a314eb (table test-table [id=786a9bedac3f40269bff250978dcfb2e]) successfully deleted
W20260811 17:19:34.510581 17678 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.157.129:41561 (error 108)
I20260811 17:19:34.510696 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 18007
I20260811 17:19:34.518451 18455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:34.518094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:41561 (local address 127.0.0.1:51966)
0811 17:19:34.518261 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:34.518345 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.129:41561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 17:19:34.518707 17678 catalog_manager.cc:4584] TS 383beaca2dc8414cb0dfbb6440c831b0 (127.9.157.129:41561): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 67978fc439014de2a0c3087842a314eb: Network error: Client connection negotiation failed: client connection to 127.9.157.129:41561: connect: Connection refused (error 111)
I20260811 17:19:34.524344 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 18211
I20260811 17:19:34.538841 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 17660
2026-08-11T17:19:34Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1153 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 17:19:34.570122 9846 mini_oidc.cc:152] Starting JWKS server
I20260811 17:19:34.572033 9846 webserver.cc:460] Webserver started at https://127.0.0.1:34491/ using document root <none> and password file <none>
I20260811 17:19:34.572184 9846 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 17:19:34.572429 9846 webserver.cc:460] Webserver started at http://127.0.0.1:37779/ using document root <none> and password file <none>
2026-08-11T17:19:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:19:34Z Disabled control of system clock
I20260811 17:19:34.584224 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:32887
--webserver_interface=127.9.157.190
--webserver_port=0
--builtin_ntp_servers=127.9.157.148:46807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:32887
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34491/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468743878666-9846-0/testchainca.cert with env {}
W20260811 17:19:34.737927 18503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:34.738240 18503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:34.738284 18503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:34.742616 18503 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:19:34.742689 18503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:34.742705 18503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:34.742729 18503 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:19:34.742744 18503 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 17:19:34.742761 18503 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 17:19:34.742774 18503 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34491/jwks.json
I20260811 17:19:34.748787 18503 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:46807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:32887
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:32887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34491/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468743878666-9846-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:34.750311 18503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:34.751339 18503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:34.760598 18512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:34.760598 18517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:34.760619 18514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:34.760985 18503 server_base.cc:1034] running on GCE node
I20260811 17:19:34.761759 18503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:34.763105 18503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:34.764335 18503 hybrid_clock.cc:648] HybridClock initialized: now 1786468774764292 us; error 57 us; skew 500 ppm
I20260811 17:19:34.766602 18503 webserver.cc:460] Webserver started at http://127.9.157.190:42573/ using document root <none> and password file <none>
I20260811 17:19:34.767266 18503 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:34.767373 18503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:34.767623 18503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:34.769971 18503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "96d96bc4fc7d4928899ff118d88d46e5"
format_stamp: "Formatted at 2026-08-11 17:19:34 on dist-test-slave-mjjr"
server_key: "f13db50b4a6926973db47fd2c65a28ad"
server_key_iv: "4a8f1543411f405620457cc95845ff1d"
server_key_version: "encryptionkey@0"
I20260811 17:19:34.770643 18503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "96d96bc4fc7d4928899ff118d88d46e5"
format_stamp: "Formatted at 2026-08-11 17:19:34 on dist-test-slave-mjjr"
server_key: "f13db50b4a6926973db47fd2c65a28ad"
server_key_iv: "4a8f1543411f405620457cc95845ff1d"
server_key_version: "encryptionkey@0"
I20260811 17:19:34.774938 18503 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 17:19:34.777642 18527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:34.779033 18503 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 17:19:34.779192 18503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "96d96bc4fc7d4928899ff118d88d46e5"
format_stamp: "Formatted at 2026-08-11 17:19:34 on dist-test-slave-mjjr"
server_key: "f13db50b4a6926973db47fd2c65a28ad"
server_key_iv: "4a8f1543411f405620457cc95845ff1d"
server_key_version: "encryptionkey@0"
I20260811 17:19:34.779321 18503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:34.800865 18503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:34.801718 18503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:34.801968 18503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:34.810550 18503 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:32887
I20260811 17:19:34.810575 18599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:32887 every 8 connection(s)
I20260811 17:19:34.811743 18503 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:19:34.813835 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 18503
I20260811 17:19:34.813946 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:19:34.814286 9846 external_mini_cluster.cc:1428] Setting key db179f2160430cbd179e55f8ec700287
I20260811 17:19:34.816102 18604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:34.824165 18604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96d96bc4fc7d4928899ff118d88d46e5: Bootstrap starting.
I20260811 17:19:34.826305 18608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:34.815945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37376 (local address 127.9.157.190:32887)
0811 17:19:34.816777 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:34.816801 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:34.816850 (+ 49us) server_negotiation.cc:407] Connection header received
0811 17:19:34.817765 (+ 915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:34.817790 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:34.818357 (+ 567us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:34.818781 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:34.819270 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:34.820845 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:34.822064 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:34.822803 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:34.823004 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:34.824004 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:34.824010 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:34.824029 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:19:34.824583 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:34.824657 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:19:34.824686 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:34.824787 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:34.824981 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 17:19:34.825271 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":163,"threads_started":1}
I20260811 17:19:34.827350 18604 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96d96bc4fc7d4928899ff118d88d46e5: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:34.828256 18604 log.cc:824] T 00000000000000000000000000000000 P 96d96bc4fc7d4928899ff118d88d46e5: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:34.829763 9846 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:19:34.830792 18604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96d96bc4fc7d4928899ff118d88d46e5: No bootstrap required, opened a new log
I20260811 17:19:34.834600 18604 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96d96bc4fc7d4928899ff118d88d46e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96d96bc4fc7d4928899ff118d88d46e5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 32887 } }
I20260811 17:19:34.834851 18604 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96d96bc4fc7d4928899ff118d88d46e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:34.834896 18604 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96d96bc4fc7d4928899ff118d88d46e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96d96bc4fc7d4928899ff118d88d46e5, State: Initialized, Role: FOLLOWER
I20260811 17:19:34.835381 18604 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96d96bc4fc7d4928899ff118d88d46e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96d96bc4fc7d4928899ff118d88d46e5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 32887 } }
I20260811 17:19:34.835515 18604 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96d96bc4fc7d4928899ff118d88d46e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:19:34.835565 18604 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96d96bc4fc7d4928899ff118d88d46e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:19:34.835695 18604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96d96bc4fc7d4928899ff118d88d46e5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:34.836731 18604 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96d96bc4fc7d4928899ff118d88d46e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96d96bc4fc7d4928899ff118d88d46e5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 32887 } }
I20260811 17:19:34.837110 18604 leader_election.cc:302] T 00000000000000000000000000000000 P 96d96bc4fc7d4928899ff118d88d46e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96d96bc4fc7d4928899ff118d88d46e5; no voters:
I20260811 17:19:34.837539 18604 leader_election.cc:288] T 00000000000000000000000000000000 P 96d96bc4fc7d4928899ff118d88d46e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:19:34.837666 18610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96d96bc4fc7d4928899ff118d88d46e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:34.837945 18610 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96d96bc4fc7d4928899ff118d88d46e5 [term 1 LEADER]: Becoming Leader. State: Replica: 96d96bc4fc7d4928899ff118d88d46e5, State: Running, Role: LEADER
I20260811 17:19:34.838300 18610 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96d96bc4fc7d4928899ff118d88d46e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96d96bc4fc7d4928899ff118d88d46e5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 32887 } }
I20260811 17:19:34.838985 18604 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96d96bc4fc7d4928899ff118d88d46e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:19:34.840122 18615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96d96bc4fc7d4928899ff118d88d46e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96d96bc4fc7d4928899ff118d88d46e5. Latest consensus state: current_term: 1 leader_uuid: "96d96bc4fc7d4928899ff118d88d46e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96d96bc4fc7d4928899ff118d88d46e5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 32887 } } }
I20260811 17:19:34.840268 18615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96d96bc4fc7d4928899ff118d88d46e5 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:34.840664 18625 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:19:34.840660 18611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96d96bc4fc7d4928899ff118d88d46e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96d96bc4fc7d4928899ff118d88d46e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96d96bc4fc7d4928899ff118d88d46e5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 32887 } } }
I20260811 17:19:34.841612 18611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96d96bc4fc7d4928899ff118d88d46e5 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:34.844220 18625 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:19:34.850348 18625 catalog_manager.cc:1347] Generated new cluster ID: 7b1ca5de34d14d09a5fdeb62ec211294
I20260811 17:19:34.850466 18625 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:19:34.867753 18625 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:19:34.868790 18625 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:19:34.881521 18625 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96d96bc4fc7d4928899ff118d88d46e5: Generated new TSK 0
I20260811 17:19:34.882356 18625 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 17:19:34.913080 18648 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 17:19:34.913323 18648 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 17:19:34.914036 18608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:34.894224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37392 (local address 127.9.157.190:32887)
0811 17:19:34.894370 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:34.894375 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:34.894618 (+ 243us) server_negotiation.cc:407] Connection header received
0811 17:19:34.894923 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:34.894929 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:34.895006 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:34.895123 (+ 117us) server_negotiation.cc:226] Negotiated authn=JWT
0811 17:19:34.895428 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:34.896037 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:34.896936 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:34.897151 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:34.897261 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:34.897662 (+ 401us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 17:19:34.913521 (+ 15859us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34491/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 17:19:34.913881 (+ 360us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37392: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34491/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 17:19:34.914119 18608 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37392: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34491/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 17:19:34.915467 18647 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:34.894129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:32887 (local address 127.0.0.1:37392)
0811 17:19:34.894521 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:34.894534 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:34.894697 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:34.895142 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:34.895145 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:34.895154 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0811 17:19:34.895304 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:34.895309 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:34.896153 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:34.896160 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:34.896803 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:34.896810 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:34.897275 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:34.897279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:34.897391 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:34.897441 (+ 50us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 17:19:34.913796 (+ 16355us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34491/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 17:19:34.914064 (+ 268us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.157.190:32887: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34491/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":117,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.9.157.190:32887: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34491/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 17:19:34.916420 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 18503
2026-08-11T17:19:34Z chronyd exiting
I20260811 17:19:34.960691 9846 test_util.cc:175] -----------------------------------------------
I20260811 17:19:34.960799 9846 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786468743878666-9846-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (392 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 17:19:34.961993 9846 mini_oidc.cc:152] Starting JWKS server
I20260811 17:19:34.962919 9846 webserver.cc:460] Webserver started at https://127.0.0.1:39679/ using document root <none> and password file <none>
I20260811 17:19:34.963008 9846 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 17:19:34.963371 9846 webserver.cc:460] Webserver started at http://127.0.0.1:34091/ using document root <none> and password file <none>
2026-08-11T17:19:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:19:34Z Disabled control of system clock
I20260811 17:19:34.975039 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:38711
--webserver_interface=127.9.157.190
--webserver_port=0
--builtin_ntp_servers=127.9.157.148:34171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:38711
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39679/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468743878666-9846-0/testchainca.cert with env {}
W20260811 17:19:35.094630 18657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:35.094945 18657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:35.094988 18657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:35.099188 18657 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:19:35.099275 18657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:35.099291 18657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:35.099308 18657 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:19:35.099323 18657 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 17:19:35.099339 18657 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 17:19:35.099354 18657 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39679/jwks.json
I20260811 17:19:35.105048 18657 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:34171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:38711
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:38711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39679/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468743878666-9846-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:35.106194 18657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:35.107252 18657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:35.116253 18666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:35.116261 18664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:35.116385 18662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:35.116518 18657 server_base.cc:1034] running on GCE node
I20260811 17:19:35.117273 18657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:35.118523 18657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:35.119753 18657 hybrid_clock.cc:648] HybridClock initialized: now 1786468775119717 us; error 76 us; skew 500 ppm
I20260811 17:19:35.121906 18657 webserver.cc:460] Webserver started at http://127.9.157.190:43359/ using document root <none> and password file <none>
I20260811 17:19:35.122545 18657 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:35.122609 18657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:35.122902 18657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:35.125226 18657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "c27bdd068fa04cfd8f10a0cab5718393"
format_stamp: "Formatted at 2026-08-11 17:19:35 on dist-test-slave-mjjr"
server_key: "b970738d706f7d53602db0a938980639"
server_key_iv: "b187487bde7b8e712211428d25d561b1"
server_key_version: "encryptionkey@0"
I20260811 17:19:35.125887 18657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "c27bdd068fa04cfd8f10a0cab5718393"
format_stamp: "Formatted at 2026-08-11 17:19:35 on dist-test-slave-mjjr"
server_key: "b970738d706f7d53602db0a938980639"
server_key_iv: "b187487bde7b8e712211428d25d561b1"
server_key_version: "encryptionkey@0"
I20260811 17:19:35.130002 18657 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:19:35.132714 18677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:35.134061 18657 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:35.134225 18657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "c27bdd068fa04cfd8f10a0cab5718393"
format_stamp: "Formatted at 2026-08-11 17:19:35 on dist-test-slave-mjjr"
server_key: "b970738d706f7d53602db0a938980639"
server_key_iv: "b187487bde7b8e712211428d25d561b1"
server_key_version: "encryptionkey@0"
I20260811 17:19:35.134361 18657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:35.172465 18657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:35.173273 18657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:35.173506 18657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:35.181334 18657 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:38711
I20260811 17:19:35.181341 18759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:38711 every 8 connection(s)
I20260811 17:19:35.182416 18657 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:19:35.185961 18761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:35.192292 18761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c27bdd068fa04cfd8f10a0cab5718393: Bootstrap starting.
I20260811 17:19:35.192757 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 18657
I20260811 17:19:35.192853 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:19:35.193137 9846 external_mini_cluster.cc:1428] Setting key 935a59a75a4557794a079a8312b22c13
I20260811 17:19:35.195410 18761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c27bdd068fa04cfd8f10a0cab5718393: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:35.196343 18761 log.cc:824] T 00000000000000000000000000000000 P c27bdd068fa04cfd8f10a0cab5718393: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:35.198602 18761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c27bdd068fa04cfd8f10a0cab5718393: No bootstrap required, opened a new log
I20260811 17:19:35.202623 18761 raft_consensus.cc:348] T 00000000000000000000000000000000 P c27bdd068fa04cfd8f10a0cab5718393 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27bdd068fa04cfd8f10a0cab5718393" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 38711 } }
I20260811 17:19:35.202965 18761 raft_consensus.cc:374] T 00000000000000000000000000000000 P c27bdd068fa04cfd8f10a0cab5718393 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:35.203058 18761 raft_consensus.cc:729] T 00000000000000000000000000000000 P c27bdd068fa04cfd8f10a0cab5718393 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c27bdd068fa04cfd8f10a0cab5718393, State: Initialized, Role: FOLLOWER
I20260811 17:19:35.203394 18766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:35.194641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47118 (local address 127.9.157.190:38711)
0811 17:19:35.195369 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:35.195386 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:35.195423 (+ 37us) server_negotiation.cc:407] Connection header received
0811 17:19:35.196263 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:35.196286 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:35.196780 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:35.197153 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:35.197623 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:35.198622 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:35.199588 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:35.200172 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:35.200359 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:35.201274 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:35.201280 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:35.201299 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:19:35.201724 (+ 425us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:35.201785 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:19:35.201816 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:35.201933 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:35.202057 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 17:19:35.202358 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":152,"threads_started":1}
I20260811 17:19:35.203681 18761 consensus_queue.cc:260] T 00000000000000000000000000000000 P c27bdd068fa04cfd8f10a0cab5718393 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27bdd068fa04cfd8f10a0cab5718393" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 38711 } }
I20260811 17:19:35.203882 18761 raft_consensus.cc:388] T 00000000000000000000000000000000 P c27bdd068fa04cfd8f10a0cab5718393 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:19:35.203962 18761 raft_consensus.cc:482] T 00000000000000000000000000000000 P c27bdd068fa04cfd8f10a0cab5718393 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:19:35.204072 18761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c27bdd068fa04cfd8f10a0cab5718393 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:35.205317 18761 raft_consensus.cc:504] T 00000000000000000000000000000000 P c27bdd068fa04cfd8f10a0cab5718393 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27bdd068fa04cfd8f10a0cab5718393" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 38711 } }
I20260811 17:19:35.205778 18761 leader_election.cc:302] T 00000000000000000000000000000000 P c27bdd068fa04cfd8f10a0cab5718393 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c27bdd068fa04cfd8f10a0cab5718393; no voters:
I20260811 17:19:35.206280 18761 leader_election.cc:288] T 00000000000000000000000000000000 P c27bdd068fa04cfd8f10a0cab5718393 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:19:35.206410 18768 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c27bdd068fa04cfd8f10a0cab5718393 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:35.206658 18768 raft_consensus.cc:686] T 00000000000000000000000000000000 P c27bdd068fa04cfd8f10a0cab5718393 [term 1 LEADER]: Becoming Leader. State: Replica: c27bdd068fa04cfd8f10a0cab5718393, State: Running, Role: LEADER
I20260811 17:19:35.207038 18768 consensus_queue.cc:237] T 00000000000000000000000000000000 P c27bdd068fa04cfd8f10a0cab5718393 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27bdd068fa04cfd8f10a0cab5718393" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 38711 } }
I20260811 17:19:35.207185 9846 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:19:35.207885 18761 sys_catalog.cc:564] T 00000000000000000000000000000000 P c27bdd068fa04cfd8f10a0cab5718393 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:19:35.209107 18769 sys_catalog.cc:455] T 00000000000000000000000000000000 P c27bdd068fa04cfd8f10a0cab5718393 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c27bdd068fa04cfd8f10a0cab5718393. Latest consensus state: current_term: 1 leader_uuid: "c27bdd068fa04cfd8f10a0cab5718393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27bdd068fa04cfd8f10a0cab5718393" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 38711 } } }
I20260811 17:19:35.209268 18769 sys_catalog.cc:458] T 00000000000000000000000000000000 P c27bdd068fa04cfd8f10a0cab5718393 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:35.209614 18770 sys_catalog.cc:455] T 00000000000000000000000000000000 P c27bdd068fa04cfd8f10a0cab5718393 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c27bdd068fa04cfd8f10a0cab5718393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27bdd068fa04cfd8f10a0cab5718393" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 38711 } } }
I20260811 17:19:35.209709 18779 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:19:35.209712 18770 sys_catalog.cc:458] T 00000000000000000000000000000000 P c27bdd068fa04cfd8f10a0cab5718393 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:35.213243 18779 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:19:35.219745 18779 catalog_manager.cc:1347] Generated new cluster ID: 846be28e736f4ca0ad807f5ed213a3da
I20260811 17:19:35.219841 18779 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:19:35.234380 18779 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:19:35.235455 18779 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:19:35.240360 18779 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c27bdd068fa04cfd8f10a0cab5718393: Generated new TSK 0
I20260811 17:19:35.241130 18779 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 17:19:35.266597 18808 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 17:19:35.266714 18808 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 17:19:35.267671 18766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:35.250763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47124 (local address 127.9.157.190:38711)
0811 17:19:35.250915 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:35.250921 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:35.251045 (+ 124us) server_negotiation.cc:407] Connection header received
0811 17:19:35.251212 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:35.251216 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:35.251286 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:35.251415 (+ 129us) server_negotiation.cc:226] Negotiated authn=JWT
0811 17:19:35.251653 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:35.252578 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:35.253498 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:35.253904 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:35.254022 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:35.254358 (+ 336us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 17:19:35.267083 (+ 12725us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39679/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 17:19:35.267457 (+ 374us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47124: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39679/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 17:19:35.267765 18766 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47124: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39679/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 17:19:35.267777 18806 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:35.250650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:38711 (local address 127.0.0.1:47124)
0811 17:19:35.250934 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:35.250957 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:35.251057 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:35.251415 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:35.251417 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:35.251423 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0811 17:19:35.251519 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:35.251523 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:35.252733 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:35.252736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:35.253369 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:35.253376 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:35.254044 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:35.254048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:35.254122 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:35.254149 (+ 27us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 17:19:35.267374 (+ 13225us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39679/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 17:19:35.267543 (+ 169us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.157.190:38711: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39679/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":111,"threads_started":1}
I20260811 17:19:35.268782 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 18657
2026-08-11T17:19:35Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (358 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 17:19:35.320547 9846 mini_oidc.cc:152] Starting JWKS server
I20260811 17:19:35.321472 9846 webserver.cc:460] Webserver started at https://127.0.0.1:45709/ using document root <none> and password file <none>
I20260811 17:19:35.321611 9846 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 17:19:35.321893 9846 webserver.cc:460] Webserver started at http://127.0.0.1:36391/ using document root <none> and password file <none>
2026-08-11T17:19:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:19:35Z Disabled control of system clock
I20260811 17:19:35.333124 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:46817
--webserver_interface=127.9.157.190
--webserver_port=0
--builtin_ntp_servers=127.9.157.148:43565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:46817
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45709/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 17:19:35.452800 18817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:35.453171 18817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:35.453258 18817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:35.457459 18817 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:19:35.457587 18817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:35.457617 18817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:35.457657 18817 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:19:35.457695 18817 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 17:19:35.457724 18817 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 17:19:35.457755 18817 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45709/jwks.json
I20260811 17:19:35.462838 18817 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:43565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:46817
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:46817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45709/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:35.464185 18817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:35.465732 18817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:35.475003 18823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:35.475003 18824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:35.475005 18826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:35.475323 18817 server_base.cc:1034] running on GCE node
I20260811 17:19:35.476132 18817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:35.477380 18817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:35.478590 18817 hybrid_clock.cc:648] HybridClock initialized: now 1786468775478560 us; error 43 us; skew 500 ppm
I20260811 17:19:35.480862 18817 webserver.cc:460] Webserver started at http://127.9.157.190:34341/ using document root <none> and password file <none>
I20260811 17:19:35.481609 18817 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:35.481714 18817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:35.481971 18817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:35.484252 18817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "287da23e43864551a67fe8dbe6927e6f"
format_stamp: "Formatted at 2026-08-11 17:19:35 on dist-test-slave-mjjr"
server_key: "05f13dee8c30e0c5126032298ae226aa"
server_key_iv: "9971d0bcb35d48b3d198fb782c581531"
server_key_version: "encryptionkey@0"
I20260811 17:19:35.484874 18817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "287da23e43864551a67fe8dbe6927e6f"
format_stamp: "Formatted at 2026-08-11 17:19:35 on dist-test-slave-mjjr"
server_key: "05f13dee8c30e0c5126032298ae226aa"
server_key_iv: "9971d0bcb35d48b3d198fb782c581531"
server_key_version: "encryptionkey@0"
I20260811 17:19:35.489140 18817 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:19:35.491995 18837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:35.493403 18817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 17:19:35.493571 18817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "287da23e43864551a67fe8dbe6927e6f"
format_stamp: "Formatted at 2026-08-11 17:19:35 on dist-test-slave-mjjr"
server_key: "05f13dee8c30e0c5126032298ae226aa"
server_key_iv: "9971d0bcb35d48b3d198fb782c581531"
server_key_version: "encryptionkey@0"
I20260811 17:19:35.493709 18817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:35.510376 18817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:35.511250 18817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:35.511495 18817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:35.520702 18817 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:46817
I20260811 17:19:35.520725 18923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:46817 every 8 connection(s)
I20260811 17:19:35.521798 18817 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:19:35.525532 18924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:35.530499 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 18817
I20260811 17:19:35.530715 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:19:35.531049 9846 external_mini_cluster.cc:1428] Setting key 2fdb17c4a61acaef384a1803a0c80c80
I20260811 17:19:35.532871 18924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 287da23e43864551a67fe8dbe6927e6f: Bootstrap starting.
I20260811 17:19:35.535495 18924 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 287da23e43864551a67fe8dbe6927e6f: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:35.536434 18924 log.cc:824] T 00000000000000000000000000000000 P 287da23e43864551a67fe8dbe6927e6f: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:35.538659 18924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 287da23e43864551a67fe8dbe6927e6f: No bootstrap required, opened a new log
I20260811 17:19:35.541817 18928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:35.532924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49770 (local address 127.9.157.190:46817)
0811 17:19:35.533708 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:35.533726 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:35.533789 (+ 63us) server_negotiation.cc:407] Connection header received
0811 17:19:35.534622 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:35.534644 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:35.535115 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:35.535522 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:35.536117 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:35.537186 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:35.538175 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:35.538679 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:35.538879 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:35.539901 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:35.539907 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:35.539922 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:19:35.540374 (+ 452us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:35.540431 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:19:35.540451 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:35.540568 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:35.540732 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 17:19:35.541026 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":168,"threads_started":1}
I20260811 17:19:35.542876 18924 raft_consensus.cc:348] T 00000000000000000000000000000000 P 287da23e43864551a67fe8dbe6927e6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "287da23e43864551a67fe8dbe6927e6f" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 46817 } }
I20260811 17:19:35.543169 18924 raft_consensus.cc:374] T 00000000000000000000000000000000 P 287da23e43864551a67fe8dbe6927e6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:35.543252 18924 raft_consensus.cc:729] T 00000000000000000000000000000000 P 287da23e43864551a67fe8dbe6927e6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 287da23e43864551a67fe8dbe6927e6f, State: Initialized, Role: FOLLOWER
I20260811 17:19:35.543771 18924 consensus_queue.cc:260] T 00000000000000000000000000000000 P 287da23e43864551a67fe8dbe6927e6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "287da23e43864551a67fe8dbe6927e6f" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 46817 } }
I20260811 17:19:35.543952 18924 raft_consensus.cc:388] T 00000000000000000000000000000000 P 287da23e43864551a67fe8dbe6927e6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:19:35.544032 18924 raft_consensus.cc:482] T 00000000000000000000000000000000 P 287da23e43864551a67fe8dbe6927e6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:19:35.544142 18924 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 287da23e43864551a67fe8dbe6927e6f [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:35.545241 9846 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:19:35.545451 18924 raft_consensus.cc:504] T 00000000000000000000000000000000 P 287da23e43864551a67fe8dbe6927e6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "287da23e43864551a67fe8dbe6927e6f" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 46817 } }
I20260811 17:19:35.545863 18924 leader_election.cc:302] T 00000000000000000000000000000000 P 287da23e43864551a67fe8dbe6927e6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 287da23e43864551a67fe8dbe6927e6f; no voters:
I20260811 17:19:35.546281 18924 leader_election.cc:288] T 00000000000000000000000000000000 P 287da23e43864551a67fe8dbe6927e6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:19:35.546446 18930 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 287da23e43864551a67fe8dbe6927e6f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:35.546729 18930 raft_consensus.cc:686] T 00000000000000000000000000000000 P 287da23e43864551a67fe8dbe6927e6f [term 1 LEADER]: Becoming Leader. State: Replica: 287da23e43864551a67fe8dbe6927e6f, State: Running, Role: LEADER
I20260811 17:19:35.547086 18930 consensus_queue.cc:237] T 00000000000000000000000000000000 P 287da23e43864551a67fe8dbe6927e6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "287da23e43864551a67fe8dbe6927e6f" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 46817 } }
I20260811 17:19:35.547945 18924 sys_catalog.cc:564] T 00000000000000000000000000000000 P 287da23e43864551a67fe8dbe6927e6f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:19:35.548405 18930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 287da23e43864551a67fe8dbe6927e6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 287da23e43864551a67fe8dbe6927e6f. Latest consensus state: current_term: 1 leader_uuid: "287da23e43864551a67fe8dbe6927e6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "287da23e43864551a67fe8dbe6927e6f" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 46817 } } }
I20260811 17:19:35.548543 18930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 287da23e43864551a67fe8dbe6927e6f [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:35.548905 18934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 287da23e43864551a67fe8dbe6927e6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "287da23e43864551a67fe8dbe6927e6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "287da23e43864551a67fe8dbe6927e6f" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 46817 } } }
I20260811 17:19:35.549067 18934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 287da23e43864551a67fe8dbe6927e6f [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:35.550863 18937 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:19:35.554491 18937 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:19:35.561544 18937 catalog_manager.cc:1347] Generated new cluster ID: 7a1089c95dd94b55921974f5c1369165
I20260811 17:19:35.561650 18937 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:19:35.590749 18937 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:19:35.592141 18937 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:19:35.597319 18937 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 287da23e43864551a67fe8dbe6927e6f: Generated new TSK 0
I20260811 17:19:35.598222 18937 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:19:35.617648 18928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:35.611958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49786 (local address 127.9.157.190:46817)
0811 17:19:35.612127 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:35.612132 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:35.612301 (+ 169us) server_negotiation.cc:407] Connection header received
0811 17:19:35.612504 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:35.612506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:35.612561 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:35.612663 (+ 102us) server_negotiation.cc:226] Negotiated authn=JWT
0811 17:19:35.612996 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:35.613583 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:35.614898 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:35.615168 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:35.615279 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:35.615766 (+ 487us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 17:19:35.617198 (+ 1432us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45709/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 17:19:35.617518 (+ 320us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49786: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45709/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 17:19:35.617760 18928 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49786: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45709/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 17:19:35.620486 18964 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:35.611870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:46817 (local address 127.0.0.1:49786)
0811 17:19:35.612215 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:35.612232 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:35.612363 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:35.612691 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:35.612695 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:35.612704 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0811 17:19:35.612842 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:35.612849 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:35.613710 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:35.613714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:35.614721 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:35.614735 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:35.615320 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:35.615325 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:35.615428 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:35.615471 (+ 43us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 17:19:35.617397 (+ 1926us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45709/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 17:19:35.620308 (+ 2911us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.157.190:46817: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45709/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":2643,"thread_start_us":125,"threads_started":1}
I20260811 17:19:35.621344 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 18817
E20260811 17:19:35.635303 18965 webserver.cc:533] Webserver: sslize failed:
E20260811 17:19:35.635408 18965 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-11T17:19:35Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (351 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T17:19:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:19:35Z Disabled control of system clock
I20260811 17:19:35.683195 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:41715
--webserver_interface=127.9.157.190
--webserver_port=0
--builtin_ntp_servers=127.9.157.148:33431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:41715
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468743878666-9846-0/testchainca.cert with env {}
W20260811 17:19:35.804538 18977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:35.804857 18977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:35.804909 18977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:35.809271 18977 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:19:35.809396 18977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:35.809418 18977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:35.809437 18977 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:19:35.809478 18977 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 17:19:35.809518 18977 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 17:19:35.809537 18977 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 17:19:35.815470 18977 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:41715
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:41715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468743878666-9846-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:35.816833 18977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:35.817948 18977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:35.828106 18986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:35.828224 18977 server_base.cc:1034] running on GCE node
W20260811 17:19:35.828075 18985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:35.828075 18988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:35.829052 18977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:35.830394 18977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:35.831698 18977 hybrid_clock.cc:648] HybridClock initialized: now 1786468775831564 us; error 97 us; skew 500 ppm
I20260811 17:19:35.838636 18977 webserver.cc:460] Webserver started at http://127.9.157.190:36151/ using document root <none> and password file <none>
I20260811 17:19:35.839399 18977 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:35.839520 18977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:35.839846 18977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:35.842250 18977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "c34c5816b7f844efa45778719806e567"
format_stamp: "Formatted at 2026-08-11 17:19:35 on dist-test-slave-mjjr"
server_key: "7cbb77e560ac6607648cf34d882f4a93"
server_key_iv: "be41def0f089718a7a303cab2a220bfe"
server_key_version: "encryptionkey@0"
I20260811 17:19:35.842905 18977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "c34c5816b7f844efa45778719806e567"
format_stamp: "Formatted at 2026-08-11 17:19:35 on dist-test-slave-mjjr"
server_key: "7cbb77e560ac6607648cf34d882f4a93"
server_key_iv: "be41def0f089718a7a303cab2a220bfe"
server_key_version: "encryptionkey@0"
I20260811 17:19:35.847937 18977 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 17:19:35.850805 19000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:35.852262 18977 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:35.852459 18977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "c34c5816b7f844efa45778719806e567"
format_stamp: "Formatted at 2026-08-11 17:19:35 on dist-test-slave-mjjr"
server_key: "7cbb77e560ac6607648cf34d882f4a93"
server_key_iv: "be41def0f089718a7a303cab2a220bfe"
server_key_version: "encryptionkey@0"
I20260811 17:19:35.852610 18977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:35.863188 18977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:35.864096 18977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:35.864372 18977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:35.874861 19077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:41715 every 8 connection(s)
I20260811 17:19:35.874861 18977 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:41715
I20260811 17:19:35.876174 18977 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:19:35.879595 19078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:35.880698 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 18977
I20260811 17:19:35.880859 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:19:35.881155 9846 external_mini_cluster.cc:1428] Setting key 56915dcf4a864c2d4ea6d967a20560b9
I20260811 17:19:35.886286 19078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c34c5816b7f844efa45778719806e567: Bootstrap starting.
I20260811 17:19:35.888937 19078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c34c5816b7f844efa45778719806e567: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:35.889706 19078 log.cc:824] T 00000000000000000000000000000000 P c34c5816b7f844efa45778719806e567: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:35.890913 19082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:35.882746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58108 (local address 127.9.157.190:41715)
0811 17:19:35.883463 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:35.883479 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:35.883519 (+ 40us) server_negotiation.cc:407] Connection header received
0811 17:19:35.884334 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:35.884353 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:35.884752 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:35.885050 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:35.885408 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:35.886350 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:35.887296 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:35.887857 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:35.888047 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:35.888946 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:35.888951 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:35.888965 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:19:35.889405 (+ 440us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:35.889468 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:19:35.889490 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:35.889604 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:35.889765 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 17:19:35.890172 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":150,"threads_started":1}
I20260811 17:19:35.891850 19078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c34c5816b7f844efa45778719806e567: No bootstrap required, opened a new log
I20260811 17:19:35.893965 9846 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:19:35.896476 19078 raft_consensus.cc:348] T 00000000000000000000000000000000 P c34c5816b7f844efa45778719806e567 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34c5816b7f844efa45778719806e567" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41715 } }
I20260811 17:19:35.896704 19078 raft_consensus.cc:374] T 00000000000000000000000000000000 P c34c5816b7f844efa45778719806e567 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:35.896739 19078 raft_consensus.cc:729] T 00000000000000000000000000000000 P c34c5816b7f844efa45778719806e567 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c34c5816b7f844efa45778719806e567, State: Initialized, Role: FOLLOWER
I20260811 17:19:35.897118 19078 consensus_queue.cc:260] T 00000000000000000000000000000000 P c34c5816b7f844efa45778719806e567 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34c5816b7f844efa45778719806e567" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41715 } }
I20260811 17:19:35.897229 19078 raft_consensus.cc:388] T 00000000000000000000000000000000 P c34c5816b7f844efa45778719806e567 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:19:35.897274 19078 raft_consensus.cc:482] T 00000000000000000000000000000000 P c34c5816b7f844efa45778719806e567 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:19:35.897374 19078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c34c5816b7f844efa45778719806e567 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:35.898401 19078 raft_consensus.cc:504] T 00000000000000000000000000000000 P c34c5816b7f844efa45778719806e567 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34c5816b7f844efa45778719806e567" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41715 } }
I20260811 17:19:35.898718 19078 leader_election.cc:302] T 00000000000000000000000000000000 P c34c5816b7f844efa45778719806e567 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c34c5816b7f844efa45778719806e567; no voters:
I20260811 17:19:35.899010 19078 leader_election.cc:288] T 00000000000000000000000000000000 P c34c5816b7f844efa45778719806e567 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:19:35.899161 19084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c34c5816b7f844efa45778719806e567 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:35.899390 19084 raft_consensus.cc:686] T 00000000000000000000000000000000 P c34c5816b7f844efa45778719806e567 [term 1 LEADER]: Becoming Leader. State: Replica: c34c5816b7f844efa45778719806e567, State: Running, Role: LEADER
I20260811 17:19:35.900126 19084 consensus_queue.cc:237] T 00000000000000000000000000000000 P c34c5816b7f844efa45778719806e567 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34c5816b7f844efa45778719806e567" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41715 } }
I20260811 17:19:35.900478 19078 sys_catalog.cc:564] T 00000000000000000000000000000000 P c34c5816b7f844efa45778719806e567 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:19:35.902057 19088 sys_catalog.cc:455] T 00000000000000000000000000000000 P c34c5816b7f844efa45778719806e567 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c34c5816b7f844efa45778719806e567. Latest consensus state: current_term: 1 leader_uuid: "c34c5816b7f844efa45778719806e567" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34c5816b7f844efa45778719806e567" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41715 } } }
I20260811 17:19:35.902032 19085 sys_catalog.cc:455] T 00000000000000000000000000000000 P c34c5816b7f844efa45778719806e567 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c34c5816b7f844efa45778719806e567" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34c5816b7f844efa45778719806e567" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41715 } } }
I20260811 17:19:35.902253 19085 sys_catalog.cc:458] T 00000000000000000000000000000000 P c34c5816b7f844efa45778719806e567 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:35.902253 19088 sys_catalog.cc:458] T 00000000000000000000000000000000 P c34c5816b7f844efa45778719806e567 [sys.catalog]: This master's current role is: LEADER
W20260811 17:19:35.905586 19104 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c34c5816b7f844efa45778719806e567: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:19:35.905683 19104 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:19:35.905773 19101 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:19:35.906541 19101 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:19:35.912576 19101 catalog_manager.cc:1347] Generated new cluster ID: bf4304fd03af4c95a281792a7980c531
I20260811 17:19:35.912664 19101 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:19:35.940151 19101 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:19:35.941555 19101 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:19:35.948683 19101 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c34c5816b7f844efa45778719806e567: Generated new TSK 0
I20260811 17:19:35.949633 19101 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:19:36.053874 19082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:35.966097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58114 (local address 127.9.157.190:41715)
0811 17:19:35.966260 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:35.966268 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:35.966459 (+ 191us) server_negotiation.cc:407] Connection header received
0811 17:19:35.966679 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:35.966682 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:35.966739 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:35.966839 (+ 100us) server_negotiation.cc:226] Negotiated authn=JWT
0811 17:19:35.967120 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:35.968277 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:35.969588 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:35.969811 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:35.969886 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:35.970242 (+ 356us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 17:19:36.053429 (+ 83187us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 17:19:36.053737 (+ 308us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58114: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 17:19:36.054003 19082 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58114: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 17:19:36.054162 19119 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:35.965990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:41715 (local address 127.0.0.1:58114)
0811 17:19:35.966347 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:35.966363 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:35.966510 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:35.966867 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:35.966870 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:35.966879 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0811 17:19:35.967005 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:35.967012 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:35.968410 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:35.968415 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:35.969450 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:35.969462 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:35.969929 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:35.969933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:35.970024 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:35.970058 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 17:19:36.053718 (+ 83660us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 17:19:36.053945 (+ 227us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.157.190:41715: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":115,"threads_started":1}
I20260811 17:19:36.054957 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 18977
2026-08-11T17:19:36Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (414 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (140 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (138 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:19:36 dist-test-slave-mjjr krb5kdc[19146](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:19:36 dist-test-slave-mjjr krb5kdc[19146](info): set up 2 sockets
Aug 11 17:19:36 dist-test-slave-mjjr krb5kdc[19146](info): commencing operation
krb5kdc: starting...
W20260811 17:19:37.668423 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.303s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:19:37 dist-test-slave-mjjr krb5kdc[19146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:19:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:19:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:19:37.809412 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:43041
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:42185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:43041
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:37.934013 19169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:37.934369 19169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:37.934461 19169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:37.938724 19169 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:19:37.938838 19169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:37.938879 19169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:37.938936 19169 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:19:37.938973 19169 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:19:37.944698 19169 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:42185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:43041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:43041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:37.945907 19169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:37.947031 19169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:37.956209 19176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:37.956236 19169 server_base.cc:1034] running on GCE node
W20260811 17:19:37.956233 19179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:37.956442 19182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:37.957168 19169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:37.958287 19169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:37.959477 19169 hybrid_clock.cc:648] HybridClock initialized: now 1786468777959448 us; error 40 us; skew 500 ppm
Aug 11 17:19:37 dist-test-slave-mjjr krb5kdc[19146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468777, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:37.962517 19169 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:19:37.963485 19169 webserver.cc:460] Webserver started at http://127.9.157.190:33665/ using document root <none> and password file <none>
I20260811 17:19:37.964143 19169 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:37.964236 19169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:37.964476 19169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:37.966729 19169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "345c9f25f277458aa4b44b4590a171c5"
format_stamp: "Formatted at 2026-08-11 17:19:37 on dist-test-slave-mjjr"
server_key: "3227ae443ee50e27dc33b88a70d4325d"
server_key_iv: "831a92200386854f779309b652394d77"
server_key_version: "encryptionkey@0"
I20260811 17:19:37.967344 19169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "345c9f25f277458aa4b44b4590a171c5"
format_stamp: "Formatted at 2026-08-11 17:19:37 on dist-test-slave-mjjr"
server_key: "3227ae443ee50e27dc33b88a70d4325d"
server_key_iv: "831a92200386854f779309b652394d77"
server_key_version: "encryptionkey@0"
I20260811 17:19:37.971765 19169 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260811 17:19:37.974658 19195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:37.976162 19169 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:37.976336 19169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "345c9f25f277458aa4b44b4590a171c5"
format_stamp: "Formatted at 2026-08-11 17:19:37 on dist-test-slave-mjjr"
server_key: "3227ae443ee50e27dc33b88a70d4325d"
server_key_iv: "831a92200386854f779309b652394d77"
server_key_version: "encryptionkey@0"
I20260811 17:19:37.976471 19169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:37.987531 19169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:37.991129 19169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:37.991379 19169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:37.999281 19169 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:43041
I20260811 17:19:37.999303 19268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:43041 every 8 connection(s)
I20260811 17:19:38.000428 19169 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:19:38.003557 19272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:38.008282 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 19169
I20260811 17:19:38.008452 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:19:38.008776 9846 external_mini_cluster.cc:1428] Setting key 180d846e14cf240df61992a05afe1877
I20260811 17:19:38.010485 19272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 345c9f25f277458aa4b44b4590a171c5: Bootstrap starting.
I20260811 17:19:38.013900 19272 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 345c9f25f277458aa4b44b4590a171c5: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:38.014984 19272 log.cc:824] T 00000000000000000000000000000000 P 345c9f25f277458aa4b44b4590a171c5: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:38.017232 19272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 345c9f25f277458aa4b44b4590a171c5: No bootstrap required, opened a new log
Aug 11 17:19:38 dist-test-slave-mjjr krb5kdc[19146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:38.021128 19272 raft_consensus.cc:348] T 00000000000000000000000000000000 P 345c9f25f277458aa4b44b4590a171c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345c9f25f277458aa4b44b4590a171c5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43041 } }
I20260811 17:19:38.021509 19272 raft_consensus.cc:374] T 00000000000000000000000000000000 P 345c9f25f277458aa4b44b4590a171c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:38.021596 19272 raft_consensus.cc:729] T 00000000000000000000000000000000 P 345c9f25f277458aa4b44b4590a171c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 345c9f25f277458aa4b44b4590a171c5, State: Initialized, Role: FOLLOWER
I20260811 17:19:38.022073 19272 consensus_queue.cc:260] T 00000000000000000000000000000000 P 345c9f25f277458aa4b44b4590a171c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345c9f25f277458aa4b44b4590a171c5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43041 } }
I20260811 17:19:38.022279 19272 raft_consensus.cc:388] T 00000000000000000000000000000000 P 345c9f25f277458aa4b44b4590a171c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:19:38.022374 19272 raft_consensus.cc:482] T 00000000000000000000000000000000 P 345c9f25f277458aa4b44b4590a171c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:19:38.022478 19272 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 345c9f25f277458aa4b44b4590a171c5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:38.023466 19272 raft_consensus.cc:504] T 00000000000000000000000000000000 P 345c9f25f277458aa4b44b4590a171c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345c9f25f277458aa4b44b4590a171c5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43041 } }
I20260811 17:19:38.023873 19272 leader_election.cc:302] T 00000000000000000000000000000000 P 345c9f25f277458aa4b44b4590a171c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 345c9f25f277458aa4b44b4590a171c5; no voters:
I20260811 17:19:38.024236 19272 leader_election.cc:288] T 00000000000000000000000000000000 P 345c9f25f277458aa4b44b4590a171c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:19:38.024375 19278 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 345c9f25f277458aa4b44b4590a171c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:38.024648 19278 raft_consensus.cc:686] T 00000000000000000000000000000000 P 345c9f25f277458aa4b44b4590a171c5 [term 1 LEADER]: Becoming Leader. State: Replica: 345c9f25f277458aa4b44b4590a171c5, State: Running, Role: LEADER
I20260811 17:19:38.025012 19278 consensus_queue.cc:237] T 00000000000000000000000000000000 P 345c9f25f277458aa4b44b4590a171c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345c9f25f277458aa4b44b4590a171c5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43041 } }
I20260811 17:19:38.025790 19272 sys_catalog.cc:564] T 00000000000000000000000000000000 P 345c9f25f277458aa4b44b4590a171c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:19:38.026644 19276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:38.010411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39900 (local address 127.9.157.190:43041)
0811 17:19:38.010936 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:38.010959 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:38.011007 (+ 48us) server_negotiation.cc:407] Connection header received
0811 17:19:38.012041 (+ 1034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:38.012070 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:38.012571 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:38.013029 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:38.013787 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:38.015514 (+ 1727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:38.016577 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:38.017107 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:38.017314 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:38.020226 (+ 2912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:38.020232 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:38.020248 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:38.020280 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:38.022512 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:38.023110 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:38.023116 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:38.023124 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:38.023262 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:38.023601 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:38.023606 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:38.023608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:38.024240 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:38.024575 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:38.025068 (+ 493us) server_negotiation.cc:299] Negotiation successful
0811 17:19:38.025415 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":161,"threads_started":1}
I20260811 17:19:38.026996 19280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 345c9f25f277458aa4b44b4590a171c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "345c9f25f277458aa4b44b4590a171c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345c9f25f277458aa4b44b4590a171c5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43041 } } }
I20260811 17:19:38.027123 19280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 345c9f25f277458aa4b44b4590a171c5 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:38.027406 19281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 345c9f25f277458aa4b44b4590a171c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 345c9f25f277458aa4b44b4590a171c5. Latest consensus state: current_term: 1 leader_uuid: "345c9f25f277458aa4b44b4590a171c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "345c9f25f277458aa4b44b4590a171c5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43041 } } }
I20260811 17:19:38.027500 19281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 345c9f25f277458aa4b44b4590a171c5 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:38.027820 19301 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:19:38.031684 19301 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:19:38.038120 19301 catalog_manager.cc:1347] Generated new cluster ID: 2833448c5f4a46e59dc88b60fa1eae5b
I20260811 17:19:38.038228 19301 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:19:38.066206 19301 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:19:38.067294 19301 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:19:38.083216 19301 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 345c9f25f277458aa4b44b4590a171c5: Generated new TSK 0
I20260811 17:19:38.084079 19301 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
I20260811 17:19:38.127735 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:0
--local_ip_for_outbound_sockets=127.9.157.129
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:43041
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:42185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:38.258834 19319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:38.259160 19319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:38.259207 19319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:38.263489 19319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:38.263585 19319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:38.263725 19319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:19:38.269654 19319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:42185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:43041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:38.270870 19319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:38.271972 19319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:38.282418 19325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:38.282425 19329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:38.282450 19331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:38.282794 19319 server_base.cc:1034] running on GCE node
I20260811 17:19:38.283288 19319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:38.284092 19319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:38.285318 19319 hybrid_clock.cc:648] HybridClock initialized: now 1786468778285283 us; error 38 us; skew 500 ppm
Aug 11 17:19:38 dist-test-slave-mjjr krb5kdc[19146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468778, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:38.288419 19319 init.cc:377] Logged in from keytab as kudu/127.9.157.129@KRBTEST.COM (short username kudu)
I20260811 17:19:38.289477 19319 webserver.cc:460] Webserver started at http://127.9.157.129:39909/ using document root <none> and password file <none>
I20260811 17:19:38.290119 19319 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:38.290220 19319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:38.290469 19319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:38.292798 19319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-0/data/instance:
uuid: "7e624b1b458a40369864d189b2ce2b44"
format_stamp: "Formatted at 2026-08-11 17:19:38 on dist-test-slave-mjjr"
server_key: "c3e13d5d5691c7d97628f3e16956a9b1"
server_key_iv: "cd0ac1e3b6c5d15ebfd6ffb908275ced"
server_key_version: "encryptionkey@0"
I20260811 17:19:38.293414 19319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance:
uuid: "7e624b1b458a40369864d189b2ce2b44"
format_stamp: "Formatted at 2026-08-11 17:19:38 on dist-test-slave-mjjr"
server_key: "c3e13d5d5691c7d97628f3e16956a9b1"
server_key_iv: "cd0ac1e3b6c5d15ebfd6ffb908275ced"
server_key_version: "encryptionkey@0"
I20260811 17:19:38.298007 19319 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:19:38.300776 19344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:38.302193 19319 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:38.302392 19319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "7e624b1b458a40369864d189b2ce2b44"
format_stamp: "Formatted at 2026-08-11 17:19:38 on dist-test-slave-mjjr"
server_key: "c3e13d5d5691c7d97628f3e16956a9b1"
server_key_iv: "cd0ac1e3b6c5d15ebfd6ffb908275ced"
server_key_version: "encryptionkey@0"
I20260811 17:19:38.302543 19319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:38.339392 19319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:38.343364 19319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:38.343627 19319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:38.344432 19319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:38.345410 19319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:38.345492 19319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:38.345570 19319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:38.345629 19319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:38.356019 19319 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:43217
I20260811 17:19:38.356084 19535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:43217 every 8 connection(s)
I20260811 17:19:38.357091 19319 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:19:38.357455 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 19319
I20260811 17:19:38.357597 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance
I20260811 17:19:38.357941 9846 external_mini_cluster.cc:1428] Setting key e9cb17777cbbedf35c02d9cb437c839b
Aug 11 17:19:38 dist-test-slave-mjjr krb5kdc[19146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468778, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.157.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.130@KRBTEST.COM" created.
I20260811 17:19:38.374965 19276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:38.359141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:37563 (local address 127.9.157.190:43041)
0811 17:19:38.359310 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:38.359314 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:38.360439 (+ 1125us) server_negotiation.cc:407] Connection header received
0811 17:19:38.362038 (+ 1599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:38.362045 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:38.362117 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:38.362196 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:38.364109 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:38.364697 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:38.366760 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:38.367023 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:38.367129 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:38.370434 (+ 3305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:38.370438 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:38.370440 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:38.370470 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:38.372242 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:38.372897 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:38.372901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:38.372902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:38.372988 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:38.373483 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:38.373486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:38.373487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:38.373759 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:38.373891 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:38.374525 (+ 634us) server_negotiation.cc:299] Negotiation successful
0811 17:19:38.374709 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 17:19:38.375866 19541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:38.359709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43041 (local address 127.9.157.129:37563)
0811 17:19:38.360245 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:38.360305 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:38.361717 (+ 1412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:38.362446 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:38.362457 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:38.363039 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:38.363867 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:38.363893 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:38.364877 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:38.364884 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:38.366594 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:38.366610 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:38.367224 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:38.367231 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:38.367512 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:38.368249 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:38.368273 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:38.370169 (+ 1896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:38.372405 (+ 2236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:38.372413 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:38.372435 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:38.372757 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:38.373091 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:38.373094 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:38.373096 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:38.373316 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:38.373913 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:38.373919 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:38.374209 (+ 290us) client_negotiation.cc:769] Sending connection context
0811 17:19:38.374410 (+ 201us) client_negotiation.cc:240] Negotiation successful
0811 17:19:38.374753 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":153,"threads_started":1}
I20260811 17:19:38.377303 19538 heartbeater.cc:347] Connected to a master server at 127.9.157.190:43041
I20260811 17:19:38.377600 19538 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:38.378271 19538 heartbeater.cc:511] Master 127.9.157.190:43041 requested a full tablet report, sending...
I20260811 17:19:38.380245 19222 ts_manager.cc:194] Registered new tserver with Master: 7e624b1b458a40369864d189b2ce2b44 (127.9.157.129:43217)
I20260811 17:19:38.381525 19222 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.129@KRBTEST.COM'} at 127.9.157.129:37563
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
I20260811 17:19:38.390866 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:0
--local_ip_for_outbound_sockets=127.9.157.130
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:43041
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:42185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:38.509153 19545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:38.509429 19545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:38.509473 19545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:38.513437 19545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:38.513495 19545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:38.513553 19545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:19:38.518898 19545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:42185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:43041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:38.519989 19545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:38.520983 19545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:38.531105 19562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:38.531141 19555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:38.531165 19556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:38.531328 19545 server_base.cc:1034] running on GCE node
I20260811 17:19:38.531950 19545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:38.532644 19545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:38.533842 19545 hybrid_clock.cc:648] HybridClock initialized: now 1786468778533828 us; error 28 us; skew 500 ppm
Aug 11 17:19:38 dist-test-slave-mjjr krb5kdc[19146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468778, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:38.536648 19545 init.cc:377] Logged in from keytab as kudu/127.9.157.130@KRBTEST.COM (short username kudu)
I20260811 17:19:38.537606 19545 webserver.cc:460] Webserver started at http://127.9.157.130:38959/ using document root <none> and password file <none>
I20260811 17:19:38.538262 19545 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:38.538357 19545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:38.538635 19545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:38.540805 19545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-1/data/instance:
uuid: "6d867f7d50f845a6a3cf25ec6b0556d8"
format_stamp: "Formatted at 2026-08-11 17:19:38 on dist-test-slave-mjjr"
server_key: "2085bdd1e4401a2a175fd8382eb1c26c"
server_key_iv: "6d8cc733228bc3e9b54868936196b7d2"
server_key_version: "encryptionkey@0"
I20260811 17:19:38.541375 19545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance:
uuid: "6d867f7d50f845a6a3cf25ec6b0556d8"
format_stamp: "Formatted at 2026-08-11 17:19:38 on dist-test-slave-mjjr"
server_key: "2085bdd1e4401a2a175fd8382eb1c26c"
server_key_iv: "6d8cc733228bc3e9b54868936196b7d2"
server_key_version: "encryptionkey@0"
I20260811 17:19:38.545261 19545 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:19:38.547830 19574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:38.549094 19545 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 17:19:38.549240 19545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "6d867f7d50f845a6a3cf25ec6b0556d8"
format_stamp: "Formatted at 2026-08-11 17:19:38 on dist-test-slave-mjjr"
server_key: "2085bdd1e4401a2a175fd8382eb1c26c"
server_key_iv: "6d8cc733228bc3e9b54868936196b7d2"
server_key_version: "encryptionkey@0"
I20260811 17:19:38.549364 19545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:38.574599 19545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:38.578970 19545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:38.579265 19545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:38.579993 19545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:38.580909 19545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:38.580996 19545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:38.581064 19545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:38.581095 19545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:38.590890 19545 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:46595
I20260811 17:19:38.590917 19742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:46595 every 8 connection(s)
I20260811 17:19:38.591957 19545 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:19:38.599195 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 19545
I20260811 17:19:38.599308 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance
I20260811 17:19:38.599619 9846 external_mini_cluster.cc:1428] Setting key 0aaf97fbce6a30003d75f212049be846
Aug 11 17:19:38 dist-test-slave-mjjr krb5kdc[19146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468778, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:38.608147 19276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:38.593938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:59923 (local address 127.9.157.190:43041)
0811 17:19:38.594099 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:38.594103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:38.594975 (+ 872us) server_negotiation.cc:407] Connection header received
0811 17:19:38.596537 (+ 1562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:38.596541 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:38.596604 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:38.596716 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:38.598007 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:38.598567 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:38.600308 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:38.600609 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:38.600710 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:38.603982 (+ 3272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:38.603986 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:38.603989 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:38.604018 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:38.605547 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:38.606137 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:38.606143 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:38.606146 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:38.606237 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:38.606644 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:38.606647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:38.606649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:38.606869 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:38.606982 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:38.607715 (+ 733us) server_negotiation.cc:299] Negotiation successful
0811 17:19:38.607889 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 17:19:38.609002 19746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:38.594321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43041 (local address 127.9.157.130:59923)
0811 17:19:38.594808 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:38.594855 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:38.596300 (+ 1445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:38.596917 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:38.596924 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:38.597312 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:38.597831 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:38.597845 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:38.598850 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:38.598853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:38.600108 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:38.600124 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:38.600811 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:38.600817 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:38.601033 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:38.601845 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:38.601864 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:38.603769 (+ 1905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:38.605687 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:38.605693 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:38.605713 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:38.606032 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:38.606334 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:38.606337 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:38.606339 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:38.606535 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:38.606981 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:38.606990 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:38.607350 (+ 360us) client_negotiation.cc:769] Sending connection context
0811 17:19:38.607562 (+ 212us) client_negotiation.cc:240] Negotiation successful
0811 17:19:38.607953 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":132,"threads_started":1}
I20260811 17:19:38.610381 19743 heartbeater.cc:347] Connected to a master server at 127.9.157.190:43041
I20260811 17:19:38.610688 19743 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:38.611402 19743 heartbeater.cc:511] Master 127.9.157.190:43041 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.157.131@KRBTEST.COM; defaulting to no policy
I20260811 17:19:38.612828 19222 ts_manager.cc:194] Registered new tserver with Master: 6d867f7d50f845a6a3cf25ec6b0556d8 (127.9.157.130:46595)
I20260811 17:19:38.613452 19222 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.130@KRBTEST.COM'} at 127.9.157.130:59923
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
I20260811 17:19:38.632001 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:0
--local_ip_for_outbound_sockets=127.9.157.131
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:43041
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:42185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:38.759330 19753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:38.759689 19753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:38.759739 19753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:38.764076 19753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:38.764165 19753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:38.764231 19753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:19:38.770412 19753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:42185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:43041
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:38.771730 19753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:38.772892 19753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:38.784356 19761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:38.784402 19767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:38.784356 19763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:38.784658 19753 server_base.cc:1034] running on GCE node
I20260811 17:19:38.785344 19753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:38.786273 19753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:38.787485 19753 hybrid_clock.cc:648] HybridClock initialized: now 1786468778787464 us; error 38 us; skew 500 ppm
Aug 11 17:19:38 dist-test-slave-mjjr krb5kdc[19146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468778, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:38.791221 19753 init.cc:377] Logged in from keytab as kudu/127.9.157.131@KRBTEST.COM (short username kudu)
I20260811 17:19:38.792436 19753 webserver.cc:460] Webserver started at http://127.9.157.131:36749/ using document root <none> and password file <none>
I20260811 17:19:38.793112 19753 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:38.793221 19753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:38.793498 19753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:38.795876 19753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-2/data/instance:
uuid: "a9bb1023092847f9a48319fcecfee036"
format_stamp: "Formatted at 2026-08-11 17:19:38 on dist-test-slave-mjjr"
server_key: "29177f9d649d976592c53115e0d9715d"
server_key_iv: "74252e41e98dd289e1ed273bd3d02061"
server_key_version: "encryptionkey@0"
I20260811 17:19:38.796535 19753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance:
uuid: "a9bb1023092847f9a48319fcecfee036"
format_stamp: "Formatted at 2026-08-11 17:19:38 on dist-test-slave-mjjr"
server_key: "29177f9d649d976592c53115e0d9715d"
server_key_iv: "74252e41e98dd289e1ed273bd3d02061"
server_key_version: "encryptionkey@0"
I20260811 17:19:38.801246 19753 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:19:38.804224 19781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:38.806005 19753 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:38.806190 19753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "a9bb1023092847f9a48319fcecfee036"
format_stamp: "Formatted at 2026-08-11 17:19:38 on dist-test-slave-mjjr"
server_key: "29177f9d649d976592c53115e0d9715d"
server_key_iv: "74252e41e98dd289e1ed273bd3d02061"
server_key_version: "encryptionkey@0"
I20260811 17:19:38.806332 19753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:38.834357 19753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:38.838078 19753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:38.838352 19753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:38.839178 19753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:38.840302 19753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:38.840382 19753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:38.840459 19753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:38.840510 19753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:38.851313 19753 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:33951
I20260811 17:19:38.851325 19949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:33951 every 8 connection(s)
I20260811 17:19:38.852413 19753 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:19:38.861640 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 19753
I20260811 17:19:38.861845 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance
I20260811 17:19:38.862201 9846 external_mini_cluster.cc:1428] Setting key 033d55b74eb7bd4fb8ef1b3fcaf35b77
Aug 11 17:19:38 dist-test-slave-mjjr krb5kdc[19146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468778, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:38.867977 19276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:38.854756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:53929 (local address 127.9.157.190:43041)
0811 17:19:38.854943 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:38.854947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:38.855900 (+ 953us) server_negotiation.cc:407] Connection header received
0811 17:19:38.857437 (+ 1537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:38.857441 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:38.857507 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:38.857618 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:38.858931 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:38.859476 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:38.860702 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:38.860980 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:38.861099 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:38.863995 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:38.864005 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:38.864008 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:38.864057 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:38.865675 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:38.866250 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:38.866253 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:38.866255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:38.866351 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:38.866742 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:38.866746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:38.866748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:38.866963 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:38.867076 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:38.867633 (+ 557us) server_negotiation.cc:299] Negotiation successful
0811 17:19:38.867797 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 17:19:38.868908 19954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:38.855164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43041 (local address 127.9.157.131:53929)
0811 17:19:38.855732 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:38.855781 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:38.857183 (+ 1402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:38.857816 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:38.857825 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:38.858241 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:38.858763 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:38.858777 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:38.859618 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:38.859628 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:38.860560 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:38.860570 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:38.861153 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:38.861158 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:38.861365 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:38.862048 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:38.862067 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:38.863815 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:38.865815 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:38.865822 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:38.865840 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:38.866130 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:38.866463 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:38.866467 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:38.866469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:38.866631 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:38.867077 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:38.867086 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:38.867376 (+ 290us) client_negotiation.cc:769] Sending connection context
0811 17:19:38.867576 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 17:19:38.867977 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":138,"threads_started":1}
I20260811 17:19:38.870297 19952 heartbeater.cc:347] Connected to a master server at 127.9.157.190:43041
I20260811 17:19:38.870584 19952 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:38.871212 19952 heartbeater.cc:511] Master 127.9.157.190:43041 requested a full tablet report, sending...
I20260811 17:19:38.872568 19222 ts_manager.cc:194] Registered new tserver with Master: a9bb1023092847f9a48319fcecfee036 (127.9.157.131:33951)
I20260811 17:19:38.873157 19222 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.131@KRBTEST.COM'} at 127.9.157.131:53929
I20260811 17:19:38.877076 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/test-admin.keytab.
Aug 11 17:19:38 dist-test-slave-mjjr krb5kdc[19146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:38.897737 9846 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 17:19:38.900856 19961 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:38.899229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38704 (local address 127.9.157.129:43217)
0811 17:19:38.899543 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:38.899550 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:38.899577 (+ 27us) server_negotiation.cc:407] Connection header received
0811 17:19:38.899631 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:38.899637 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:38.899937 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:38.900072 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:38.900636 (+ 564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38704: BlockingRecv error: recv got EOF from 127.0.0.1:38704 (error 108)
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":83,"threads_started":1}
W20260811 17:19:38.901021 9846 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/krb5cc' of type FILE
Aug 11 17:19:38 dist-test-slave-mjjr krb5kdc[19146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:38.902148 9846 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 17:19:38 dist-test-slave-mjjr krb5kdc[19146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:38.903107 9846 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 17:19:38 dist-test-slave-mjjr krb5kdc[19146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.129@KRBTEST.COM
I20260811 17:19:38.914968 19961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:38.904227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38714 (local address 127.9.157.129:43217)
0811 17:19:38.904373 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:38.904379 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:38.904451 (+ 72us) server_negotiation.cc:407] Connection header received
0811 17:19:38.904632 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:38.904635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:38.904679 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:38.904767 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:38.905375 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:38.906269 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:38.907193 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:38.907734 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:38.907856 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:38.910566 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:38.910579 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:38.910597 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:38.910656 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:38.912634 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:38.913172 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:38.913188 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:38.913189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:38.913287 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:38.913663 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:38.913666 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:38.913668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:38.914008 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:38.914201 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:38.914536 (+ 335us) server_negotiation.cc:299] Negotiation successful
0811 17:19:38.914798 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 17:19:38.920297 19961 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:38.918935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38728 (local address 127.9.157.129:43217)
0811 17:19:38.919123 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:38.919128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:38.919256 (+ 128us) server_negotiation.cc:407] Connection header received
0811 17:19:38.919453 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:38.919456 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:38.919507 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:38.919607 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:38.920158 (+ 551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38728: BlockingRecv error: recv got EOF from 127.0.0.1:38728 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260811 17:19:38.920503 9846 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/krb5cc' of type FILE
Aug 11 17:19:38 dist-test-slave-mjjr krb5kdc[19146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:38.921643 9846 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 17:19:38 dist-test-slave-mjjr krb5kdc[19146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:38.922569 9846 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 17:19:38 dist-test-slave-mjjr krb5kdc[19146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.129@KRBTEST.COM
I20260811 17:19:38.934624 19961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:38.923620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38734 (local address 127.9.157.129:43217)
0811 17:19:38.923835 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:38.923842 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:38.924406 (+ 564us) server_negotiation.cc:407] Connection header received
0811 17:19:38.924548 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:38.924551 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:38.924602 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:38.924687 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:38.925433 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:38.926363 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:38.927444 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:38.927790 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:38.927906 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:38.930617 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:38.930621 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:38.930623 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:38.930656 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:38.932492 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:38.933051 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:38.933054 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:38.933055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:38.933143 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:38.933449 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:38.933453 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:38.933454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:38.933662 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:38.933782 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:38.934273 (+ 491us) server_negotiation.cc:299] Negotiation successful
0811 17:19:38.934394 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
W20260811 17:19:38.938017 19961 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:38.936684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38738 (local address 127.9.157.129:43217)
0811 17:19:38.936851 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:38.936856 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:38.937001 (+ 145us) server_negotiation.cc:407] Connection header received
0811 17:19:38.937168 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:38.937171 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:38.937217 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:38.937324 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:38.937875 (+ 551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38738: BlockingRecv error: recv got EOF from 127.0.0.1:38738 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 17:19:38.938154 9846 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786468743878666-9846-0/krb5kdc/krb5cc' of type FILE
Aug 11 17:19:38 dist-test-slave-mjjr krb5kdc[19146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:38.939402 9846 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 17:19:38 dist-test-slave-mjjr krb5kdc[19146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:38.940575 9846 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 17:19:38 dist-test-slave-mjjr krb5kdc[19146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.129@KRBTEST.COM
I20260811 17:19:38.950640 19961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:38.941608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38754 (local address 127.9.157.129:43217)
0811 17:19:38.941781 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:38.941785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:38.941838 (+ 53us) server_negotiation.cc:407] Connection header received
0811 17:19:38.941952 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:38.941954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:38.941999 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:38.942071 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:38.942570 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:38.943083 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:38.944054 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:38.944267 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:38.944351 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:38.947021 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:38.947025 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:38.947027 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:38.947059 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:38.948755 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:38.949231 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:38.949234 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:38.949236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:38.949322 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:38.949693 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:38.949696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:38.949697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:38.949937 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:38.950072 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:38.950286 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 17:19:38.950436 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 17:19:38.952090 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 19319
I20260811 17:19:38.967567 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 19545
I20260811 17:19:38.994480 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 19753
I20260811 17:19:39.011229 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 19169
2026-08-11T17:19:39Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2682 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:19:39 dist-test-slave-mjjr krb5kdc[19981](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:19:39 dist-test-slave-mjjr krb5kdc[19981](info): set up 2 sockets
Aug 11 17:19:39 dist-test-slave-mjjr krb5kdc[19981](info): commencing operation
krb5kdc: starting...
W20260811 17:19:40.290220 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:19:40 dist-test-slave-mjjr krb5kdc[19981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:19:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:19:40Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.190.keytab.
Entry for principal oryx/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.190.keytab.
I20260811 17:19:40.428470 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:34565
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.190.keytab
--principal=oryx/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:39005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:34565
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:40.550140 20003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:40.550510 20003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:40.550599 20003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:40.554986 20003 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:19:40.555100 20003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:40.555137 20003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:40.555178 20003 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:19:40.555217 20003 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:19:40.561139 20003 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:39005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:34565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.190.keytab
--principal=oryx/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:34565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:40.562470 20003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:40.563669 20003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:40.573201 20013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:40.573201 20011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:40.573237 20016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:40.573333 20003 server_base.cc:1034] running on GCE node
I20260811 17:19:40.574136 20003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:40.575464 20003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:40.576675 20003 hybrid_clock.cc:648] HybridClock initialized: now 1786468780576642 us; error 51 us; skew 500 ppm
Aug 11 17:19:40 dist-test-slave-mjjr krb5kdc[19981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468780, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:40.579783 20003 init.cc:377] Logged in from keytab as oryx/127.9.157.190@KRBTEST.COM (short username oryx)
I20260811 17:19:40.580811 20003 webserver.cc:460] Webserver started at http://127.9.157.190:33377/ using document root <none> and password file <none>
I20260811 17:19:40.581386 20003 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:40.581451 20003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:40.581634 20003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:40.583995 20003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "41918f3984d74e9a9e03e35f07467cb1"
format_stamp: "Formatted at 2026-08-11 17:19:40 on dist-test-slave-mjjr"
server_key: "7bfea14a6b519488b1dab88fc7c2b0b5"
server_key_iv: "7052c58db2ae89896169f91fea3c5e9e"
server_key_version: "encryptionkey@0"
I20260811 17:19:40.584578 20003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "41918f3984d74e9a9e03e35f07467cb1"
format_stamp: "Formatted at 2026-08-11 17:19:40 on dist-test-slave-mjjr"
server_key: "7bfea14a6b519488b1dab88fc7c2b0b5"
server_key_iv: "7052c58db2ae89896169f91fea3c5e9e"
server_key_version: "encryptionkey@0"
I20260811 17:19:40.588686 20003 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:19:40.591439 20032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:40.592885 20003 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 17:19:40.593030 20003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "41918f3984d74e9a9e03e35f07467cb1"
format_stamp: "Formatted at 2026-08-11 17:19:40 on dist-test-slave-mjjr"
server_key: "7bfea14a6b519488b1dab88fc7c2b0b5"
server_key_iv: "7052c58db2ae89896169f91fea3c5e9e"
server_key_version: "encryptionkey@0"
I20260811 17:19:40.593142 20003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:40.616010 20003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:40.620244 20003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:40.620448 20003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:40.628283 20003 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:34565
I20260811 17:19:40.628659 20104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:34565 every 8 connection(s)
I20260811 17:19:40.629462 20003 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:19:40.632831 20105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:40.637207 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 20003
I20260811 17:19:40.637387 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:19:40.637684 9846 external_mini_cluster.cc:1428] Setting key 51d48b60417bbea29bf092a5ede89a9f
I20260811 17:19:40.639241 20105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1: Bootstrap starting.
I20260811 17:19:40.642285 20105 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:40.643272 20105 log.cc:824] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:40.645627 20105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1: No bootstrap required, opened a new log
Aug 11 17:19:40 dist-test-slave-mjjr krb5kdc[19981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM
I20260811 17:19:40.650775 20105 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } }
I20260811 17:19:40.651114 20105 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:40.651190 20105 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41918f3984d74e9a9e03e35f07467cb1, State: Initialized, Role: FOLLOWER
I20260811 17:19:40.651763 20105 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } }
I20260811 17:19:40.651950 20105 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:19:40.652026 20105 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:19:40.652166 20105 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:40.653563 20105 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } }
I20260811 17:19:40.653970 20105 leader_election.cc:302] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41918f3984d74e9a9e03e35f07467cb1; no voters:
I20260811 17:19:40.654364 20105 leader_election.cc:288] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:19:40.654547 20118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:40.654816 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:40.639332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53010 (local address 127.9.157.190:34565)
0811 17:19:40.639900 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:40.639921 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:40.639968 (+ 47us) server_negotiation.cc:407] Connection header received
0811 17:19:40.640896 (+ 928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:40.640922 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:40.641368 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:40.641743 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:40.642663 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:40.644078 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:40.644972 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:40.645671 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:40.645849 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:40.648688 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:40.648697 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:40.648725 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:40.648769 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:40.651012 (+ 2243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:40.651633 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:40.651658 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:40.651665 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:40.651823 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:40.652199 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:40.652203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:40.652204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:40.652791 (+ 587us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:40.652999 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:40.653348 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 17:19:40.653700 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":204,"threads_started":1}
I20260811 17:19:40.654884 20118 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [term 1 LEADER]: Becoming Leader. State: Replica: 41918f3984d74e9a9e03e35f07467cb1, State: Running, Role: LEADER
I20260811 17:19:40.655406 20118 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } }
I20260811 17:19:40.655830 20105 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:19:40.657291 20120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41918f3984d74e9a9e03e35f07467cb1. Latest consensus state: current_term: 1 leader_uuid: "41918f3984d74e9a9e03e35f07467cb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } }
I20260811 17:19:40.657426 20120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:40.657603 20119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41918f3984d74e9a9e03e35f07467cb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } }
I20260811 17:19:40.657770 20119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:40.658066 20135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:19:40.661315 20135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:19:40.667264 20135 catalog_manager.cc:1347] Generated new cluster ID: 7725b3cbf87546e186272ad03fb69783
I20260811 17:19:40.667363 20135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:19:40.719344 20135 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:19:40.720408 20135 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:19:40.724722 20135 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1: Generated new TSK 0
I20260811 17:19:40.725479 20135 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.129.keytab.
Entry for principal oryx/127.9.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.129.keytab.
I20260811 17:19:40.763346 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:0
--local_ip_for_outbound_sockets=127.9.157.129
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:34565
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.129.keytab
--principal=oryx/127.9.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:39005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:40.885540 20148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:40.885910 20148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:40.885998 20148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:40.890378 20148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:40.890506 20148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:40.890606 20148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:19:40.896412 20148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:39005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.129.keytab
--principal=oryx/127.9.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:34565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:40.897702 20148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:40.898901 20148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:40.909802 20155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:40.909806 20158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:40.909802 20156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:40.910245 20148 server_base.cc:1034] running on GCE node
I20260811 17:19:40.910763 20148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:40.911599 20148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:40.912851 20148 hybrid_clock.cc:648] HybridClock initialized: now 1786468780912829 us; error 39 us; skew 500 ppm
Aug 11 17:19:40 dist-test-slave-mjjr krb5kdc[19981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468780, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:40.919469 20148 init.cc:377] Logged in from keytab as oryx/127.9.157.129@KRBTEST.COM (short username oryx)
I20260811 17:19:40.920672 20148 webserver.cc:460] Webserver started at http://127.9.157.129:33591/ using document root <none> and password file <none>
I20260811 17:19:40.921355 20148 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:40.921460 20148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:40.921727 20148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:40.924199 20148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/data/instance:
uuid: "6756578bcb7b438b8a5bcbaaecc2914a"
format_stamp: "Formatted at 2026-08-11 17:19:40 on dist-test-slave-mjjr"
server_key: "29ed5ac84a1ddb486e50e95f70ab7c96"
server_key_iv: "224f0f05bece7085e7856ab756c278a4"
server_key_version: "encryptionkey@0"
I20260811 17:19:40.924868 20148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance:
uuid: "6756578bcb7b438b8a5bcbaaecc2914a"
format_stamp: "Formatted at 2026-08-11 17:19:40 on dist-test-slave-mjjr"
server_key: "29ed5ac84a1ddb486e50e95f70ab7c96"
server_key_iv: "224f0f05bece7085e7856ab756c278a4"
server_key_version: "encryptionkey@0"
I20260811 17:19:40.929322 20148 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:19:40.932215 20166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:40.933645 20148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:40.933813 20148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "6756578bcb7b438b8a5bcbaaecc2914a"
format_stamp: "Formatted at 2026-08-11 17:19:40 on dist-test-slave-mjjr"
server_key: "29ed5ac84a1ddb486e50e95f70ab7c96"
server_key_iv: "224f0f05bece7085e7856ab756c278a4"
server_key_version: "encryptionkey@0"
I20260811 17:19:40.933957 20148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:40.956276 20148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:40.960202 20148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:40.960448 20148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:40.961177 20148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:40.962116 20148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:40.962191 20148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:40.962261 20148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:40.962307 20148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:40.971768 20148 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:38203
I20260811 17:19:40.971786 20334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:38203 every 8 connection(s)
I20260811 17:19:40.972774 20148 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
Aug 11 17:19:40 dist-test-slave-mjjr krb5kdc[19981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468780, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM
I20260811 17:19:40.982986 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 20148
I20260811 17:19:40.983134 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance
I20260811 17:19:40.983526 9846 external_mini_cluster.cc:1428] Setting key 03c770e26037f162447ac3755a8156bc
I20260811 17:19:40.987541 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:40.974873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:43153 (local address 127.9.157.190:34565)
0811 17:19:40.975028 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:40.975032 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:40.976145 (+ 1113us) server_negotiation.cc:407] Connection header received
0811 17:19:40.977041 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:40.977045 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:40.977095 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:40.977173 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:40.978423 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:40.979037 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:40.980272 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:40.980509 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:40.980598 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:40.983495 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:40.983498 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:40.983501 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:40.983530 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:40.985301 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:40.985885 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:40.985890 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:40.985896 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:40.985974 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:40.986353 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:40.986356 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:40.986358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:40.986567 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:40.986678 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:40.987228 (+ 550us) server_negotiation.cc:299] Negotiation successful
0811 17:19:40.987375 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 17:19:40.988533 20340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:40.975278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.9.157.129:43153)
0811 17:19:40.976016 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:40.976060 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:40.976812 (+ 752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:40.977344 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:40.977350 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:40.977740 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:40.978251 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:40.978265 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:40.979178 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:40.979180 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:40.980130 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:40.980141 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:40.980654 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:40.980656 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:40.980800 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:40.981480 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:40.981497 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:40.983271 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:40.985442 (+ 2171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:40.985449 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:40.985469 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:40.985756 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:40.986075 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:40.986078 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:40.986080 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:40.986238 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:40.986680 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:40.986689 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:40.986978 (+ 289us) client_negotiation.cc:769] Sending connection context
0811 17:19:40.987187 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 17:19:40.987554 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":118,"threads_started":1}
I20260811 17:19:40.989984 20335 heartbeater.cc:347] Connected to a master server at 127.9.157.190:34565
I20260811 17:19:40.990255 20335 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:40.990794 20335 heartbeater.cc:511] Master 127.9.157.190:34565 requested a full tablet report, sending...
I20260811 17:19:40.992515 20058 ts_manager.cc:194] Registered new tserver with Master: 6756578bcb7b438b8a5bcbaaecc2914a (127.9.157.129:38203)
I20260811 17:19:40.993700 20058 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.157.129@KRBTEST.COM'} at 127.9.157.129:43153
WARNING: no policy specified for oryx/127.9.157.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.157.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.130.keytab.
Entry for principal oryx/127.9.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.130.keytab.
I20260811 17:19:41.015468 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:0
--local_ip_for_outbound_sockets=127.9.157.130
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:34565
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.130.keytab
--principal=oryx/127.9.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:39005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:41.138820 20344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:41.139194 20344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:41.139271 20344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:41.143723 20344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:41.143800 20344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:41.143904 20344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:19:41.149691 20344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:39005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.130.keytab
--principal=oryx/127.9.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:34565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:41.150954 20344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:41.152104 20344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:41.162819 20352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:41.162819 20360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:41.162999 20344 server_base.cc:1034] running on GCE node
W20260811 17:19:41.163285 20354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:41.163802 20344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:41.164508 20344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:41.165693 20344 hybrid_clock.cc:648] HybridClock initialized: now 1786468781165644 us; error 60 us; skew 500 ppm
Aug 11 17:19:41 dist-test-slave-mjjr krb5kdc[19981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468781, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:41.168830 20344 init.cc:377] Logged in from keytab as oryx/127.9.157.130@KRBTEST.COM (short username oryx)
I20260811 17:19:41.169867 20344 webserver.cc:460] Webserver started at http://127.9.157.130:38247/ using document root <none> and password file <none>
I20260811 17:19:41.170498 20344 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:41.170598 20344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:41.170856 20344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:41.173291 20344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/data/instance:
uuid: "7432ee13358d444494d13841c91bfd5f"
format_stamp: "Formatted at 2026-08-11 17:19:41 on dist-test-slave-mjjr"
server_key: "e1aa114d69bfd56287c34baccecaaec4"
server_key_iv: "198d0e0334ea7c1c291063294891ae22"
server_key_version: "encryptionkey@0"
I20260811 17:19:41.173918 20344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance:
uuid: "7432ee13358d444494d13841c91bfd5f"
format_stamp: "Formatted at 2026-08-11 17:19:41 on dist-test-slave-mjjr"
server_key: "e1aa114d69bfd56287c34baccecaaec4"
server_key_iv: "198d0e0334ea7c1c291063294891ae22"
server_key_version: "encryptionkey@0"
I20260811 17:19:41.178088 20344 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:19:41.180907 20371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:41.182425 20344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:41.182546 20344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "7432ee13358d444494d13841c91bfd5f"
format_stamp: "Formatted at 2026-08-11 17:19:41 on dist-test-slave-mjjr"
server_key: "e1aa114d69bfd56287c34baccecaaec4"
server_key_iv: "198d0e0334ea7c1c291063294891ae22"
server_key_version: "encryptionkey@0"
I20260811 17:19:41.182657 20344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:41.208822 20344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:41.212248 20344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:41.212427 20344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:41.213030 20344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:41.213981 20344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:41.214068 20344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:41.214144 20344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:41.214191 20344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:41.223605 20344 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:37833
I20260811 17:19:41.223683 20538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:37833 every 8 connection(s)
I20260811 17:19:41.224704 20344 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:19:41.234138 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 20344
I20260811 17:19:41.234256 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance
Aug 11 17:19:41 dist-test-slave-mjjr krb5kdc[19981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468781, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM
I20260811 17:19:41.234580 9846 external_mini_cluster.cc:1428] Setting key cb803b674395ff48ade96186e4e084ee
I20260811 17:19:41.239637 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:41.226859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:56825 (local address 127.9.157.190:34565)
0811 17:19:41.227015 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:41.227019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:41.228099 (+ 1080us) server_negotiation.cc:407] Connection header received
0811 17:19:41.228991 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:41.228994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:41.229046 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:41.229129 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:41.230372 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.230926 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.232192 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.232423 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.232509 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:41.235254 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:41.235259 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:41.235261 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:41.235293 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:41.236940 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:41.237658 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:41.237665 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:41.237669 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:41.237844 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:41.238268 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:41.238271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:41.238275 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:41.238474 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:41.238595 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:41.239382 (+ 787us) server_negotiation.cc:299] Negotiation successful
0811 17:19:41.239500 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 17:19:41.240952 20544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:41.227187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.9.157.130:56825)
0811 17:19:41.227948 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:41.227993 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:41.228776 (+ 783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:41.229317 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:41.229324 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:41.229720 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:41.230225 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:41.230238 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.231054 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.231057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:41.232076 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:41.232086 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.232567 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.232570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:41.232716 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:41.233296 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:41.233313 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:41.235061 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:41.237075 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:41.237085 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:41.237108 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:41.237529 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:41.237991 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:41.237993 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:41.237995 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:41.238148 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:41.238670 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:41.238680 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:41.239075 (+ 395us) client_negotiation.cc:769] Sending connection context
0811 17:19:41.239318 (+ 243us) client_negotiation.cc:240] Negotiation successful
0811 17:19:41.239757 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":118,"threads_started":1}
I20260811 17:19:41.242301 20539 heartbeater.cc:347] Connected to a master server at 127.9.157.190:34565
I20260811 17:19:41.242630 20539 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:41.243170 20539 heartbeater.cc:511] Master 127.9.157.190:34565 requested a full tablet report, sending...
I20260811 17:19:41.244366 20058 ts_manager.cc:194] Registered new tserver with Master: 7432ee13358d444494d13841c91bfd5f (127.9.157.130:37833)
I20260811 17:19:41.244978 20058 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.157.130@KRBTEST.COM'} at 127.9.157.130:56825
WARNING: no policy specified for oryx/127.9.157.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.157.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.131.keytab.
Entry for principal oryx/127.9.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.131.keytab.
I20260811 17:19:41.267182 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:0
--local_ip_for_outbound_sockets=127.9.157.131
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:34565
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.131.keytab
--principal=oryx/127.9.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:39005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:41.391151 20547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:41.391526 20547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:41.391613 20547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:41.395949 20547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:41.396080 20547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:41.396178 20547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:19:41.402205 20547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:39005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.131.keytab
--principal=oryx/127.9.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:34565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:41.403520 20547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:41.404704 20547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:41.415313 20554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:41.415352 20552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:41.415352 20556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:41.415833 20547 server_base.cc:1034] running on GCE node
I20260811 17:19:41.416281 20547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:41.416989 20547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:41.418200 20547 hybrid_clock.cc:648] HybridClock initialized: now 1786468781418164 us; error 49 us; skew 500 ppm
Aug 11 17:19:41 dist-test-slave-mjjr krb5kdc[19981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468781, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:41.421329 20547 init.cc:377] Logged in from keytab as oryx/127.9.157.131@KRBTEST.COM (short username oryx)
I20260811 17:19:41.422369 20547 webserver.cc:460] Webserver started at http://127.9.157.131:36981/ using document root <none> and password file <none>
I20260811 17:19:41.423007 20547 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:41.423105 20547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:41.423362 20547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:41.425595 20547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/data/instance:
uuid: "4cd592e9c42a4fe8881afa489954dd8e"
format_stamp: "Formatted at 2026-08-11 17:19:41 on dist-test-slave-mjjr"
server_key: "c1f6350bd9b592680116d27a25a970d6"
server_key_iv: "9e755240c94edb9ed6bab427488b4d33"
server_key_version: "encryptionkey@0"
I20260811 17:19:41.426209 20547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance:
uuid: "4cd592e9c42a4fe8881afa489954dd8e"
format_stamp: "Formatted at 2026-08-11 17:19:41 on dist-test-slave-mjjr"
server_key: "c1f6350bd9b592680116d27a25a970d6"
server_key_iv: "9e755240c94edb9ed6bab427488b4d33"
server_key_version: "encryptionkey@0"
I20260811 17:19:41.430450 20547 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:19:41.433182 20571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:41.434494 20547 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 17:19:41.434674 20547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "4cd592e9c42a4fe8881afa489954dd8e"
format_stamp: "Formatted at 2026-08-11 17:19:41 on dist-test-slave-mjjr"
server_key: "c1f6350bd9b592680116d27a25a970d6"
server_key_iv: "9e755240c94edb9ed6bab427488b4d33"
server_key_version: "encryptionkey@0"
I20260811 17:19:41.434840 20547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:41.447103 20547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:41.450770 20547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:41.450999 20547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:41.451720 20547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:41.452629 20547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:41.452705 20547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:41.452804 20547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:41.452857 20547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:41.462718 20547 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:37867
I20260811 17:19:41.462741 20742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:37867 every 8 connection(s)
I20260811 17:19:41.463768 20547 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:19:41.465736 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 20547
I20260811 17:19:41.465894 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance
I20260811 17:19:41.466216 9846 external_mini_cluster.cc:1428] Setting key ebdc1f21f39fb8422b3cf8500f835afc
Aug 11 17:19:41 dist-test-slave-mjjr krb5kdc[19981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468781, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM
I20260811 17:19:41.478948 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:41.465770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:34715 (local address 127.9.157.190:34565)
0811 17:19:41.465944 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:41.465949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:41.467260 (+ 1311us) server_negotiation.cc:407] Connection header received
0811 17:19:41.468395 (+ 1135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:41.468399 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:41.468456 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:41.468538 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:41.469780 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.470337 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.471610 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.471921 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.472040 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:41.474920 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:41.474923 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:41.474925 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:41.474954 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:41.476676 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:41.477293 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:41.477295 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:41.477297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:41.477375 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:41.477783 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:41.477785 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:41.477786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:41.477974 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:41.478050 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:41.478660 (+ 610us) server_negotiation.cc:299] Negotiation successful
0811 17:19:41.478815 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 17:19:41.480281 20746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:41.466194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.9.157.131:34715)
0811 17:19:41.467057 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:41.467102 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:41.468169 (+ 1067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:41.468722 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:41.468730 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:41.469110 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:41.469620 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:41.469635 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.470474 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.470479 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:41.471454 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:41.471465 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.472093 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.472098 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:41.472287 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:41.473031 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:41.473061 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:41.474708 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:41.476820 (+ 2112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:41.476827 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:41.476844 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:41.477179 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:41.477474 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:41.477477 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:41.477478 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:41.477657 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:41.478087 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:41.478093 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:41.478403 (+ 310us) client_negotiation.cc:769] Sending connection context
0811 17:19:41.478607 (+ 204us) client_negotiation.cc:240] Negotiation successful
0811 17:19:41.479082 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":166,"threads_started":1}
I20260811 17:19:41.481717 20743 heartbeater.cc:347] Connected to a master server at 127.9.157.190:34565
I20260811 17:19:41.482007 20743 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:41.482575 20743 heartbeater.cc:511] Master 127.9.157.190:34565 requested a full tablet report, sending...
I20260811 17:19:41.483914 20058 ts_manager.cc:194] Registered new tserver with Master: 4cd592e9c42a4fe8881afa489954dd8e (127.9.157.131:37867)
I20260811 17:19:41.484525 20058 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.157.131@KRBTEST.COM'} at 127.9.157.131:34715
I20260811 17:19:41.493351 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.157.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.157.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.157.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.189.keytab.
Entry for principal oryx/127.9.157.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.189.keytab.
I20260811 17:19:41.525179 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.189:33385
--webserver_interface=127.9.157.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.189.keytab
--principal=oryx/127.9.157.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:39005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:34565,127.9.157.189:33385
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:41.652416 20750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:41.652781 20750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:41.653038 20750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:41.657655 20750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:19:41.657783 20750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:41.657863 20750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:41.657923 20750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:19:41.657960 20750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:19:41.663791 20750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:39005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:34565,127.9.157.189:33385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.189.keytab
--principal=oryx/127.9.157.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.189:33385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.157.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
W20260811 17:19:41.664090 20750 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.9.157.190:34565,127.9.157.189:33385'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 17:19:41.665099 20750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:41.666196 20750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:41.676012 20756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:41.676033 20757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:41.676038 20759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:41.676127 20750 server_base.cc:1034] running on GCE node
I20260811 17:19:41.676952 20750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:41.678181 20750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:41.679387 20750 hybrid_clock.cc:648] HybridClock initialized: now 1786468781679354 us; error 46 us; skew 500 ppm
Aug 11 17:19:41 dist-test-slave-mjjr krb5kdc[19981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468781, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:41.682556 20750 init.cc:377] Logged in from keytab as oryx/127.9.157.189@KRBTEST.COM (short username oryx)
I20260811 17:19:41.683614 20750 webserver.cc:460] Webserver started at http://127.9.157.189:44623/ using document root <none> and password file <none>
I20260811 17:19:41.684315 20750 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:41.684415 20750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:41.684738 20750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:41.687189 20750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-1/data/instance:
uuid: "2409126f018b4ea38b67722ebcfdc44d"
format_stamp: "Formatted at 2026-08-11 17:19:41 on dist-test-slave-mjjr"
server_key: "6af89a0eb7f787b14802dd9281e1e84f"
server_key_iv: "af5a8febf5e7ebd0be992a5c8e2cb588"
server_key_version: "encryptionkey@0"
I20260811 17:19:41.687893 20750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-1/wal/instance:
uuid: "2409126f018b4ea38b67722ebcfdc44d"
format_stamp: "Formatted at 2026-08-11 17:19:41 on dist-test-slave-mjjr"
server_key: "6af89a0eb7f787b14802dd9281e1e84f"
server_key_iv: "af5a8febf5e7ebd0be992a5c8e2cb588"
server_key_version: "encryptionkey@0"
I20260811 17:19:41.692155 20750 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 17:19:41.694831 20774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:41.696226 20750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:41.696395 20750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-1/wal
uuid: "2409126f018b4ea38b67722ebcfdc44d"
format_stamp: "Formatted at 2026-08-11 17:19:41 on dist-test-slave-mjjr"
server_key: "6af89a0eb7f787b14802dd9281e1e84f"
server_key_iv: "af5a8febf5e7ebd0be992a5c8e2cb588"
server_key_version: "encryptionkey@0"
I20260811 17:19:41.696532 20750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:41.716871 20750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:41.720299 20750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:41.720526 20750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:41.728645 20750 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.189:33385
I20260811 17:19:41.728952 20854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.189:33385 every 8 connection(s)
I20260811 17:19:41.729738 20750 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-1/data/info.pb
I20260811 17:19:41.732766 20855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:41.734163 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 20750
I20260811 17:19:41.737632 20855 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 17:19:41 dist-test-slave-mjjr krb5kdc[19981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.157.189@KRBTEST.COM
Aug 11 17:19:41 dist-test-slave-mjjr krb5kdc[19981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468781, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.189@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM
I20260811 17:19:41.749621 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:41.739775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53026 (local address 127.9.157.190:34565)
0811 17:19:41.739912 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:41.739921 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:41.740404 (+ 483us) server_negotiation.cc:407] Connection header received
0811 17:19:41.740584 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:41.740588 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:41.740645 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:41.740730 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:41.741428 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.742082 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.743178 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.743416 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.743530 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:41.746025 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:41.746029 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:41.746032 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:41.746064 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:41.747571 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:41.748105 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:41.748109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:41.748111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:41.748195 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:41.748475 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:41.748478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:41.748480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:41.748781 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:41.748908 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:41.749330 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 17:19:41.749455 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 17:19:41.750291 20861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:41.739715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.0.0.1:53026)
0811 17:19:41.740271 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:41.740313 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:41.740468 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:41.740786 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:41.740793 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:41.741165 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:41.741290 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:41.741296 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.742218 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.742222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:41.743045 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:41.743055 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.743576 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.743580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:41.743735 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:41.744313 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:41.744337 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:41.745867 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:41.747692 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:41.747706 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:41.747716 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:41.748004 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:41.748289 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:41.748291 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:41.748292 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:41.748372 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:41.748930 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:41.748938 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:41.749072 (+ 134us) client_negotiation.cc:769] Sending connection context
0811 17:19:41.749249 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 17:19:41.749457 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":190,"threads_started":1}
I20260811 17:19:41.750294 20860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:41.735952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48630 (local address 127.9.157.189:33385)
0811 17:19:41.736424 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:41.736441 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:41.736479 (+ 38us) server_negotiation.cc:407] Connection header received
0811 17:19:41.737122 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:41.737139 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:41.737573 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:41.737917 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:41.738919 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.740480 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.741370 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.741866 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.742045 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:41.744783 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:41.744789 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:41.744799 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:41.744821 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:41.746878 (+ 2057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:41.747350 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:41.747358 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:41.747367 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:41.747513 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:41.747896 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:41.747899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:41.747900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:41.748477 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:41.748700 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:41.749018 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 17:19:41.749341 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":138,"threads_started":1}
I20260811 17:19:41.751961 20855 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 17:19:41 dist-test-slave-mjjr krb5kdc[19981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468781, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.189@KRBTEST.COM for oryx/127.9.157.189@KRBTEST.COM
I20260811 17:19:41.760175 20861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:41.752526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:33385 (local address 127.0.0.1:48640)
0811 17:19:41.752626 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:41.752642 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:41.752721 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:41.752930 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:41.752932 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:41.753079 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:41.753174 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:41.753179 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.754236 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.754239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:41.754755 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:41.754762 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.755220 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.755223 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:41.755302 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:41.755816 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:41.755827 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:41.757228 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:41.758770 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:41.758773 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:41.758774 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:41.758963 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:41.759205 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:41.759207 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:41.759208 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:41.759280 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:41.759725 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:41.759728 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:41.759808 (+ 80us) client_negotiation.cc:769] Sending connection context
0811 17:19:41.759892 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 17:19:41.759997 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 17:19:41.760182 20860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:41.752603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48640 (local address 127.9.157.189:33385)
0811 17:19:41.752720 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:41.752725 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:41.752741 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:19:41.752779 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:41.752782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:41.752831 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:41.752901 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:41.753270 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.754131 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.754885 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.755119 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.755200 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:41.757389 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:41.757392 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:41.757394 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:41.757417 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:41.758669 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:41.759051 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:41.759053 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:41.759055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:41.759126 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:41.759357 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:41.759359 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:41.759361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:41.759600 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:41.759732 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:41.759957 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 17:19:41.760054 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 17:19:41.763278 20855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d: Bootstrap starting.
I20260811 17:19:41.766489 20855 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:41.767205 20855 log.cc:824] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:41.769273 20855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d: No bootstrap required, opened a new log
I20260811 17:19:41.772533 20855 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } }
I20260811 17:19:41.772746 20855 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:41.772780 20855 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2409126f018b4ea38b67722ebcfdc44d, State: Initialized, Role: FOLLOWER
I20260811 17:19:41.773181 20855 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } }
I20260811 17:19:41.773645 20866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } } }
I20260811 17:19:41.773789 20866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:19:41.774426 20855 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [sys.catalog]: configured and running, proceeding with master startup.
W20260811 17:19:41.779408 20881 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:19:41.779507 20881 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:19:41.782596 20750 master_runner.cc:428] Detected that this master 2409126f018b4ea38b67722ebcfdc44d is joining an existing cluster
I20260811 17:19:41.782677 20750 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.157.189:33385
I20260811 17:19:41.793557 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:41.784899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53030 (local address 127.9.157.190:34565)
0811 17:19:41.785314 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:41.785319 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:41.785335 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:19:41.785380 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:41.785383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:41.785432 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:41.785524 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:41.786095 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.786980 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.788191 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.788540 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.788633 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:41.789851 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:41.789855 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:41.789858 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:41.789887 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:41.791739 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:41.792166 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:41.792169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:41.792171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:41.792262 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:41.792588 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:41.792591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:41.792593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:41.792833 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:41.792947 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:41.793263 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 17:19:41.793386 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327}
I20260811 17:19:41.793586 20893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:41.784847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.0.0.1:53030)
0811 17:19:41.785168 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:41.785186 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:41.785286 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:41.785554 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:41.785560 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:41.785823 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:41.785975 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:41.785981 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.787106 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.787109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:41.788065 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:41.788077 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.788645 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.788648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:41.788737 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:41.789336 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:41.789351 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:41.789687 (+ 336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:41.791848 (+ 2161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:41.791851 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:41.791853 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:41.792066 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:41.792364 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:41.792369 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:41.792372 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:41.792479 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:41.792951 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:41.792954 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:41.793079 (+ 125us) client_negotiation.cc:769] Sending connection context
0811 17:19:41.793200 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 17:19:41.793375 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"spinlock_wait_cycles":1408,"thread_start_us":102,"threads_started":1}
I20260811 17:19:41.793819 20860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:41.785515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48648 (local address 127.9.157.189:33385)
0811 17:19:41.785629 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:41.785633 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:41.785644 (+ 11us) server_negotiation.cc:407] Connection header received
0811 17:19:41.785674 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:41.785679 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:41.785720 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:41.785803 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:41.786358 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.787078 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.788044 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.788368 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.788468 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:41.789800 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:41.789804 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:41.789809 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:41.789835 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:41.791434 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:41.791962 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:41.791966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:41.791968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:41.792057 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:41.792996 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:41.793000 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:41.793002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:41.793213 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:41.793302 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:41.793555 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 17:19:41.793665 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45,"spinlock_wait_cycles":7936}
I20260811 17:19:41.794260 20894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:41.784941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:33385 (local address 127.0.0.1:48648)
0811 17:19:41.785271 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:41.785284 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:41.785370 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:41.785866 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:41.785869 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:41.786104 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:41.786251 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:41.786259 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.787183 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.787186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:41.787928 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:41.787938 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.788509 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.788514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:41.788639 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:41.789165 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:41.789181 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:41.789647 (+ 466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:41.791559 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:41.791562 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:41.791565 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:41.791867 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:41.792383 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:41.792385 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:41.792387 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:41.792485 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:41.793334 (+ 849us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:41.793337 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:41.793410 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 17:19:41.793966 (+ 556us) client_negotiation.cc:240] Negotiation successful
0811 17:19:41.794092 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":116,"threads_started":1}
Aug 11 17:19:41 dist-test-slave-mjjr krb5kdc[19981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468780, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.190@KRBTEST.COM for oryx/127.9.157.189@KRBTEST.COM
I20260811 17:19:41.809835 20895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:41.799177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:33385 (local address 127.0.0.1:48664)
0811 17:19:41.799564 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:41.799617 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:41.799780 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:41.800152 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:41.800157 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:41.800587 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:41.800703 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:41.800709 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.801731 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.801734 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:41.802404 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:41.802413 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.803006 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.803010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:41.803139 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:41.803773 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:41.803787 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:41.805617 (+ 1830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:41.807616 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:41.807627 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:41.807635 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:41.807926 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:41.808277 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:41.808282 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:41.808287 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:41.808402 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:41.808946 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:41.808957 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:41.809115 (+ 158us) client_negotiation.cc:769] Sending connection context
0811 17:19:41.809269 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 17:19:41.809530 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":130,"threads_started":1}
I20260811 17:19:41.810667 20860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:41.799248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48664 (local address 127.9.157.189:33385)
0811 17:19:41.799390 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:41.799395 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:41.799706 (+ 311us) server_negotiation.cc:407] Connection header received
0811 17:19:41.799961 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:41.799964 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:41.800020 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:41.800110 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:41.800814 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.801613 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.802523 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.802860 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.802975 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:41.805779 (+ 2804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:41.805785 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:41.805791 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:41.805822 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:41.807490 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:41.808042 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:41.808046 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:41.808048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:41.808157 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:41.808515 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:41.808521 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:41.808525 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:41.808781 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:41.808882 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:41.810327 (+ 1445us) server_negotiation.cc:299] Negotiation successful
0811 17:19:41.810478 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 17:19:41.812850 20058 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.157.189:33385: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d"
member_type: NON_VOTER
last_known_addr {
host: "127.9.157.189"
port: 33385
}
attrs {
promote: true
}
}
dest_uuid: "41918f3984d74e9a9e03e35f07467cb1"
cas_config_opid_index: -1
I20260811 17:19:41.813719 20058 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: NON_VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: true } }
WARNING: no policy specified for oryx/127.9.157.188@KRBTEST.COM; defaulting to no policy
I20260811 17:19:41.815958 20898 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 2409126f018b4ea38b67722ebcfdc44d (127.9.157.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: NON_VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: true } } }
I20260811 17:19:41.816851 20813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:41.818228 20898 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.157.189:33385
I20260811 17:19:41.818465 20813 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [term 1 FOLLOWER]: Refusing update from remote peer 41918f3984d74e9a9e03e35f07467cb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 17:19:41.818909 20750 master.cc:537] Master@127.9.157.189:33385 shutting down...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.157.188@KRBTEST.COM" created.
I20260811 17:19:41.820025 20898 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: NON_VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:41.821297 20899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "41918f3984d74e9a9e03e35f07467cb1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: NON_VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: true } } }
W20260811 17:19:41.824061 20034 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260811 17:19:41.824463 20899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [sys.catalog]: This master's current role is: LEADER
W20260811 17:19:41.825446 20034 consensus_peers.cc:487] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 -> Peer 2409126f018b4ea38b67722ebcfdc44d (127.9.157.189:33385): Couldn't send request to peer 2409126f018b4ea38b67722ebcfdc44d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 17:19:41.825929 20750 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:19:41.826259 20750 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:41.826393 20750 tablet_replica.cc:330] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.157.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.188.keytab.
Entry for principal oryx/127.9.157.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.188.keytab.
I20260811 17:19:41.839769 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.188:44451
--webserver_interface=127.9.157.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.188.keytab
--principal=oryx/127.9.157.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:39005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:34565,127.9.157.189:33385,127.9.157.188:44451
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:19:41.844542 20750 master.cc:552] Master@127.9.157.189:33385 shutdown complete.
I20260811 17:19:41.844635 20750 master_runner.cc:305] Clearing existing system tablet
I20260811 17:19:41.844902 20750 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:41.844985 20750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:41.846253 20750 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 17:19:41.847184 20909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:41.847457 20750 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:41.847596 20750 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 17:19:41.847675 20750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-1/wal
uuid: "2409126f018b4ea38b67722ebcfdc44d"
format_stamp: "Formatted at 2026-08-11 17:19:41 on dist-test-slave-mjjr"
server_key: "6af89a0eb7f787b14802dd9281e1e84f"
server_key_iv: "af5a8febf5e7ebd0be992a5c8e2cb588"
server_key_version: "encryptionkey@0"
I20260811 17:19:41.849618 20750 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:41.851786 20750 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 17:19:41.851904 20750 log.cc:1196] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 17:19:41.852213 20750 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d: Deleting consensus metadata
I20260811 17:19:41.852428 20750 master_runner.cc:315] Copying system tablet from 127.9.157.190:34565
I20260811 17:19:41.853893 20750 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d: tablet copy: Beginning tablet copy session from remote peer at address 127.9.157.190:34565
I20260811 17:19:41.862627 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:41.854640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.0.0.1:53040)
0811 17:19:41.854982 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:41.855000 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:41.855141 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:41.855436 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:41.855438 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:41.855766 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:41.855897 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:41.855904 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.856818 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.856825 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:41.857526 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:41.857537 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.858049 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.858053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:41.858151 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:41.858786 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:41.858799 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:41.859137 (+ 338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:41.861004 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:41.861009 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:41.861011 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:41.861224 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:41.861534 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:41.861538 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:41.861540 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:41.861631 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:41.862036 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:41.862040 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:41.862149 (+ 109us) client_negotiation.cc:769] Sending connection context
0811 17:19:41.862261 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 17:19:41.862432 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":101,"threads_started":1}
I20260811 17:19:41.862617 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:41.854742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53040 (local address 127.9.157.190:34565)
0811 17:19:41.854902 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:41.854907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:41.855086 (+ 179us) server_negotiation.cc:407] Connection header received
0811 17:19:41.855253 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:41.855256 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:41.855310 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:41.855414 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:41.856027 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.856697 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.857673 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:41.857922 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:41.858041 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:41.859287 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:41.859291 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:41.859294 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:41.859321 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:41.860880 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:41.861346 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:41.861350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:41.861352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:41.861436 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:41.861730 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:41.861734 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:41.861736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:41.861931 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:41.862043 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:41.862322 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 17:19:41.862439 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 17:19:41.863847 20084 tablet_copy_service.cc:140] P 41918f3984d74e9a9e03e35f07467cb1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 2409126f018b4ea38b67722ebcfdc44d ({username='oryx', principal='oryx/127.9.157.189@KRBTEST.COM'} at 127.0.0.1:53040)
I20260811 17:19:41.864065 20084 tablet_copy_service.cc:161] P 41918f3984d74e9a9e03e35f07467cb1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 2409126f018b4ea38b67722ebcfdc44d at {username='oryx', principal='oryx/127.9.157.189@KRBTEST.COM'} at 127.0.0.1:53040: session id = 2409126f018b4ea38b67722ebcfdc44d-00000000000000000000000000000000
I20260811 17:19:41.866770 20084 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 17:19:41.871495 20750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 17:19:41.874764 20750 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d: tablet copy: Starting download of 0 data blocks...
I20260811 17:19:41.875017 20750 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d: tablet copy: Starting download of 1 WAL segments...
I20260811 17:19:41.877511 20750 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 17:19:41.879562 20084 tablet_copy_service.cc:342] P 41918f3984d74e9a9e03e35f07467cb1: Request end of tablet copy session 2409126f018b4ea38b67722ebcfdc44d-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.157.189@KRBTEST.COM'} at 127.0.0.1:53040
I20260811 17:19:41.879760 20084 tablet_copy_service.cc:433] P 41918f3984d74e9a9e03e35f07467cb1: ending tablet copy session 2409126f018b4ea38b67722ebcfdc44d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 2409126f018b4ea38b67722ebcfdc44d
I20260811 17:19:41.888729 20750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:41.890594 20924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:41.890684 20917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:41.890851 20918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:41.891685 20750 server_base.cc:1034] running on GCE node
I20260811 17:19:41.891886 20750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:41.892272 20750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:41.893445 20750 hybrid_clock.cc:648] HybridClock initialized: now 1786468781893424 us; error 38 us; skew 500 ppm
Aug 11 17:19:41 dist-test-slave-mjjr krb5kdc[19981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468781, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:41.895107 20750 init.cc:377] Logged in from keytab as oryx/127.9.157.189@KRBTEST.COM (short username oryx)
I20260811 17:19:41.895695 20750 webserver.cc:460] Webserver started at http://127.9.157.189:40759/ using document root <none> and password file <none>
I20260811 17:19:41.895928 20750 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:41.895989 20750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:41.897449 20750 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 17:19:41.898630 20931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:41.898872 20750 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 17:19:41.898975 20750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-1/wal
uuid: "2409126f018b4ea38b67722ebcfdc44d"
format_stamp: "Formatted at 2026-08-11 17:19:41 on dist-test-slave-mjjr"
server_key: "6af89a0eb7f787b14802dd9281e1e84f"
server_key_iv: "af5a8febf5e7ebd0be992a5c8e2cb588"
server_key_version: "encryptionkey@0"
I20260811 17:19:41.899089 20750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:41.928690 20750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:41.930922 20750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:41.936483 20750 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.189:33385
I20260811 17:19:41.936510 21008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.189:33385 every 8 connection(s)
I20260811 17:19:41.937186 20750 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-1/data/info.pb
I20260811 17:19:41.941112 21009 sys_catalog.cc:263] Verifying existing consensus state
I20260811 17:19:41.942067 21009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d: Bootstrap starting.
I20260811 17:19:41.952023 21009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 17:19:41.952418 21009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d: Bootstrap complete.
I20260811 17:19:41.953045 21009 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: NON_VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: true } }
I20260811 17:19:41.953197 21009 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2409126f018b4ea38b67722ebcfdc44d, State: Initialized, Role: LEARNER
I20260811 17:19:41.953353 21009 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: NON_VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: true } }
I20260811 17:19:41.953780 21013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: NON_VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: true } } }
I20260811 17:19:41.953959 21013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [sys.catalog]: This master's current role is: LEARNER
I20260811 17:19:41.953965 21009 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:19:41.958143 21035 catalog_manager.cc:1259] Loaded cluster ID: 7725b3cbf87546e186272ad03fb69783
I20260811 17:19:41.958247 21035 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d: loading cluster ID for follower catalog manager: success
I20260811 17:19:41.960186 21035 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d: acquiring CA information for follower catalog manager: success
I20260811 17:19:41.961436 21035 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 17:19:41.996548 20335 heartbeater.cc:503] Master 127.9.157.190:34565 was elected leader, sending a full tablet report...
W20260811 17:19:42.007582 20905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:42.007913 20905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:42.007957 20905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:42.012136 20905 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:19:42.012207 20905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:42.012228 20905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:42.012245 20905 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:19:42.012261 20905 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:19:42.017822 20905 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:39005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:34565,127.9.157.189:33385,127.9.157.188:44451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.188.keytab
--principal=oryx/127.9.157.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.188:44451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.157.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:42.019084 20905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:42.020251 20905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:42.029744 21047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:42.029747 21045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:42.029841 20905 server_base.cc:1034] running on GCE node
W20260811 17:19:42.029744 21050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:42.030826 20905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:42.032100 20905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:42.033301 20905 hybrid_clock.cc:648] HybridClock initialized: now 1786468782033271 us; error 41 us; skew 500 ppm
Aug 11 17:19:42 dist-test-slave-mjjr krb5kdc[19981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468782, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:42.036475 20905 init.cc:377] Logged in from keytab as oryx/127.9.157.188@KRBTEST.COM (short username oryx)
I20260811 17:19:42.037457 20905 webserver.cc:460] Webserver started at http://127.9.157.188:36701/ using document root <none> and password file <none>
I20260811 17:19:42.038097 20905 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:42.038225 20905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:42.038507 20905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:42.041075 20905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-2/data/instance:
uuid: "bac5ec8af44844e388ff31729b5549bc"
format_stamp: "Formatted at 2026-08-11 17:19:42 on dist-test-slave-mjjr"
server_key: "0ede314c7c1d4ec35b1b99d7c71f3e88"
server_key_iv: "c64efe91831abc501f9d2d292ab75a3f"
server_key_version: "encryptionkey@0"
I20260811 17:19:42.041713 20905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-2/wal/instance:
uuid: "bac5ec8af44844e388ff31729b5549bc"
format_stamp: "Formatted at 2026-08-11 17:19:42 on dist-test-slave-mjjr"
server_key: "0ede314c7c1d4ec35b1b99d7c71f3e88"
server_key_iv: "c64efe91831abc501f9d2d292ab75a3f"
server_key_version: "encryptionkey@0"
I20260811 17:19:42.046140 20905 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:19:42.048954 21059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:42.050381 20905 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:19:42.050575 20905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-2/wal
uuid: "bac5ec8af44844e388ff31729b5549bc"
format_stamp: "Formatted at 2026-08-11 17:19:42 on dist-test-slave-mjjr"
server_key: "0ede314c7c1d4ec35b1b99d7c71f3e88"
server_key_iv: "c64efe91831abc501f9d2d292ab75a3f"
server_key_version: "encryptionkey@0"
I20260811 17:19:42.050729 20905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:42.064502 20905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:42.068308 20905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:42.068555 20905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:42.076920 20905 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.188:44451
I20260811 17:19:42.077236 21136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.188:44451 every 8 connection(s)
I20260811 17:19:42.078042 20905 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-2/data/info.pb
I20260811 17:19:42.079969 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 20905
I20260811 17:19:42.081672 21138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:42.086824 21138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 17:19:42 dist-test-slave-mjjr krb5kdc[19981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.157.188@KRBTEST.COM
Aug 11 17:19:42 dist-test-slave-mjjr krb5kdc[19981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468782, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.188@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM
I20260811 17:19:42.098315 21142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.081749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33698 (local address 127.9.157.188:44451)
0811 17:19:42.082274 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:42.082291 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:42.082339 (+ 48us) server_negotiation.cc:407] Connection header received
0811 17:19:42.083082 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:42.083100 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:42.083450 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:42.083875 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:42.084890 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.086122 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.087403 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.088029 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.088174 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.091373 (+ 3199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:42.091379 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:42.091389 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:42.091417 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:42.093533 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.094096 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.094107 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.094112 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.094290 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.094603 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.094607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.094609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.095189 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:42.095411 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:42.095820 (+ 409us) server_negotiation.cc:299] Negotiation successful
0811 17:19:42.096142 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":165,"threads_started":1}
I20260811 17:19:42.100106 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.089354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53050 (local address 127.9.157.190:34565)
0811 17:19:42.089518 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:42.089523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:42.089967 (+ 444us) server_negotiation.cc:407] Connection header received
0811 17:19:42.090155 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:42.090158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:42.090227 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:42.090328 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:42.091152 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.091904 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.093333 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.093562 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.093714 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.096288 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:42.096292 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:42.096294 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:42.096322 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:42.097938 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.098572 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.098580 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.098582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.098664 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.098973 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.098977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.098979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.099170 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:42.099416 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:42.099874 (+ 458us) server_negotiation.cc:299] Negotiation successful
0811 17:19:42.099987 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 17:19:42.100203 21145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.089160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.0.0.1:53050)
0811 17:19:42.089801 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:42.089861 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:42.090030 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:42.090399 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:42.090405 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:42.090837 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:42.091006 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.091013 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.092038 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.092042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.093164 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.093181 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.093721 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.093724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.093827 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.094349 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:42.094367 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:42.096128 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:42.098059 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.098070 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.098078 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.098426 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.098781 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.098784 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.098786 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.098878 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.099468 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:42.099475 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:42.099604 (+ 129us) client_negotiation.cc:769] Sending connection context
0811 17:19:42.099800 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 17:19:42.099996 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":205,"threads_started":1}
I20260811 17:19:42.101353 21138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 17:19:42 dist-test-slave-mjjr krb5kdc[19981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468782, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.188@KRBTEST.COM for oryx/127.9.157.189@KRBTEST.COM
I20260811 17:19:42.110678 21145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.101936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:33385 (local address 127.0.0.1:48666)
0811 17:19:42.102087 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:42.102102 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:42.102180 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:42.102569 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:42.102571 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:42.102731 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:42.102846 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.102852 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.103926 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.103929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.104510 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.104519 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.105028 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.105031 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.105117 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.105659 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:42.105680 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:42.107044 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:42.109031 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.109034 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.109036 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.109243 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.109606 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.109608 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.109610 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.109706 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.110212 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:42.110215 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:42.110311 (+ 96us) client_negotiation.cc:769] Sending connection context
0811 17:19:42.110409 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 17:19:42.110541 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 17:19:42.110769 21146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.102060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48666 (local address 127.9.157.189:33385)
0811 17:19:42.102353 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:42.102357 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:42.102370 (+ 13us) server_negotiation.cc:407] Connection header received
0811 17:19:42.102409 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:42.102411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:42.102450 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:42.102527 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:42.102992 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.103785 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.104634 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.104889 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.104986 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.107208 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:42.107212 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:42.107214 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:42.107242 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:42.108898 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.109341 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.109344 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.109345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.109489 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.109829 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.109832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.109833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.110101 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:42.110190 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:42.110513 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 17:19:42.110638 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":132,"threads_started":1}
I20260811 17:19:42.111887 21138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.188" port: 44451 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 17:19:42 dist-test-slave-mjjr krb5kdc[19981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468782, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.188@KRBTEST.COM for oryx/127.9.157.188@KRBTEST.COM
I20260811 17:19:42.121845 21145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.112519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:44451 (local address 127.0.0.1:33700)
0811 17:19:42.112648 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:42.112663 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:42.112751 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:42.113088 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:42.113091 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:42.113240 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:42.113328 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.113333 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.114647 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.114650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.115308 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.115320 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.115987 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.115990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.116054 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.116624 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:42.116637 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:42.118258 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:42.120310 (+ 2052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.120313 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.120314 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.120516 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.120849 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.120852 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.120853 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.120956 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.121402 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:42.121404 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:42.121496 (+ 92us) client_negotiation.cc:769] Sending connection context
0811 17:19:42.121582 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 17:19:42.121707 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 17:19:42.121989 21142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.112599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33700 (local address 127.9.157.188:44451)
0811 17:19:42.112784 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:42.112793 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:42.112809 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:19:42.112848 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:42.112851 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:42.112888 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:42.112978 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:42.113475 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.114031 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.115469 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.115827 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.115946 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.118449 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:42.118453 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:42.118455 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:42.118480 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:42.120178 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.120632 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.120636 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.120637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.120728 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.121075 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.121079 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.121080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.121301 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:42.121393 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:42.121664 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 17:19:42.121800 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 17:19:42.125453 21138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc: Bootstrap starting.
I20260811 17:19:42.128104 21138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:42.128916 21138 log.cc:824] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:42.131112 21138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc: No bootstrap required, opened a new log
I20260811 17:19:42.134407 21138 raft_consensus.cc:348] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 44451 } }
I20260811 17:19:42.134631 21138 raft_consensus.cc:374] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:42.134667 21138 raft_consensus.cc:729] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bac5ec8af44844e388ff31729b5549bc, State: Initialized, Role: FOLLOWER
I20260811 17:19:42.135157 21138 consensus_queue.cc:260] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 44451 } }
I20260811 17:19:42.135807 21148 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 44451 } } }
I20260811 17:19:42.135953 21148 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:19:42.136557 21138 sys_catalog.cc:564] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [sys.catalog]: configured and running, proceeding with master startup.
W20260811 17:19:42.142110 21163 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:19:42.142222 21163 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:19:42.146190 20905 master_runner.cc:428] Detected that this master bac5ec8af44844e388ff31729b5549bc is joining an existing cluster
I20260811 17:19:42.146286 20905 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.157.188:44451
I20260811 17:19:42.152180 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 20148
I20260811 17:19:42.180704 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.149740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53062 (local address 127.9.157.190:34565)
0811 17:19:42.149864 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:42.149870 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:42.149886 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:19:42.149942 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:42.149947 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:42.149998 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:42.150080 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:42.169065 (+ 18985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.169838 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.171842 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.172232 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.172324 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.175256 (+ 2932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:42.175262 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:42.175265 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:42.175302 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:42.177991 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.178625 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.178631 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.178633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.178751 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.179142 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.179147 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.179152 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.179474 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:42.179563 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:42.180300 (+ 737us) server_negotiation.cc:299] Negotiation successful
0811 17:19:42.180460 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 17:19:42.181447 21177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.148824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.0.0.1:53062)
0811 17:19:42.149318 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:42.149446 (+ 128us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:42.149549 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:42.150211 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:42.150215 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:42.168379 (+ 18164us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:42.168722 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.168747 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.170028 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.170035 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.171684 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.171695 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.172551 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.172555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.173155 (+ 600us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.174479 (+ 1324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:42.174497 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:42.175032 (+ 535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:42.178154 (+ 3122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.178157 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.178160 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.178469 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.178892 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.178898 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.178904 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.179012 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.179699 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:42.179706 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:42.179840 (+ 134us) client_negotiation.cc:769] Sending connection context
0811 17:19:42.181006 (+ 1166us) client_negotiation.cc:240] Negotiation successful
0811 17:19:42.181182 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":475,"thread_start_us":128,"threads_started":1}
I20260811 17:19:42.184964 21146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.148983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48670 (local address 127.9.157.189:33385)
0811 17:19:42.149116 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:42.149120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:42.149844 (+ 724us) server_negotiation.cc:407] Connection header received
0811 17:19:42.150043 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:42.150046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:42.150100 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:42.150152 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:42.173516 (+ 23364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.174288 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.175742 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.176136 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.176274 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.180296 (+ 4022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:42.180300 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:42.180303 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:42.180339 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:42.182613 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.183242 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.183246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.183248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.183358 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.183777 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.183781 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.183783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.184039 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:42.184129 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:42.184570 (+ 441us) server_negotiation.cc:299] Negotiation successful
0811 17:19:42.184759 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 17:19:42.185845 21175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.148895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:33385 (local address 127.0.0.1:48670)
0811 17:19:42.149759 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:42.149774 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:42.149930 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:42.150306 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:42.150309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:42.172753 (+ 22444us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:42.172894 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.172904 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.174495 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.174498 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.175355 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.175366 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.176514 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.176519 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.177073 (+ 554us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.179487 (+ 2414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:42.179506 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:42.180093 (+ 587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:42.182825 (+ 2732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.182828 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.182830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.183093 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.183499 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.183502 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.183504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.183600 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.184222 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:42.184228 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:42.184348 (+ 120us) client_negotiation.cc:769] Sending connection context
0811 17:19:42.185457 (+ 1109us) client_negotiation.cc:240] Negotiation successful
0811 17:19:42.185624 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":801,"thread_start_us":103,"threads_started":1}
I20260811 17:19:42.186305 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:38203
--local_ip_for_outbound_sockets=127.9.157.129
--tserver_master_addrs=127.9.157.190:34565
--webserver_port=33591
--webserver_interface=127.9.157.129
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.129.keytab
--principal=oryx/127.9.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:39005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.157.190:34565,127.9.157.189:33385
--tserver_master_addrs=127.9.157.190:34565,127.9.157.189:33385,127.9.157.188:44451 with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:19:42.187330 21142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.149387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33706 (local address 127.9.157.188:44451)
0811 17:19:42.149498 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:42.149502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:42.149930 (+ 428us) server_negotiation.cc:407] Connection header received
0811 17:19:42.150077 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:42.150080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:42.150114 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:42.150156 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:42.171697 (+ 21541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.172399 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.173530 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.173844 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.173926 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.176297 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:42.176301 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:42.176303 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:42.176348 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:42.178868 (+ 2520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.180727 (+ 1859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.180732 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.180734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.180839 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.184844 (+ 4005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.184849 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.184851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.185079 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:42.185201 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:42.187023 (+ 1822us) server_negotiation.cc:299] Negotiation successful
0811 17:19:42.187162 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 17:19:42.188647 21178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.149366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:44451 (local address 127.0.0.1:33706)
0811 17:19:42.149844 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:42.149859 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:42.149984 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:42.150278 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:42.150283 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:42.171229 (+ 20946us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:42.171471 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.171487 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.172542 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.172545 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.173397 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.173410 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.174106 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.174109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.174204 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.175593 (+ 1389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:42.175613 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:42.176128 (+ 515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:42.179073 (+ 2945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.179075 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.179079 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.179341 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.184336 (+ 4995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.184339 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.184341 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.184470 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.186284 (+ 1814us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:42.186287 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:42.186800 (+ 513us) client_negotiation.cc:769] Sending connection context
0811 17:19:42.188189 (+ 1389us) client_negotiation.cc:240] Negotiation successful
0811 17:19:42.188379 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":1,"thread_start_us":89,"threads_started":1}
Aug 11 17:19:42 dist-test-slave-mjjr krb5kdc[19981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468780, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.190@KRBTEST.COM for oryx/127.9.157.188@KRBTEST.COM
I20260811 17:19:42.196839 21142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.187004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33716 (local address 127.9.157.188:44451)
0811 17:19:42.187490 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:42.187495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:42.187511 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:19:42.187592 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:42.187605 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:42.187708 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:42.187810 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:42.188375 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.189157 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.190314 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.190600 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.190731 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.193210 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:42.193220 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:42.193222 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:42.193246 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:42.194858 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.195372 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.195377 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.195379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.195469 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.195844 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.195848 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.195850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.196065 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:42.196189 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:42.196502 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 17:19:42.196622 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":163,"threads_started":1}
I20260811 17:19:42.196879 20895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.186697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:44451 (local address 127.0.0.1:33716)
0811 17:19:42.186998 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:42.187016 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:42.187108 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:42.187836 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:42.187843 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:42.188079 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:42.188222 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.188229 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.189296 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.189300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.190170 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.190182 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.190739 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.190744 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.190851 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.191453 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:42.191467 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:42.193040 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:42.195000 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.195006 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.195008 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.195253 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.195587 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.195590 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.195592 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.195728 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.196181 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:42.196191 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:42.196310 (+ 119us) client_negotiation.cc:769] Sending connection context
0811 17:19:42.196431 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 17:19:42.196619 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202}
I20260811 17:19:42.198832 20058 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.157.188:44451: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "bac5ec8af44844e388ff31729b5549bc"
member_type: NON_VOTER
last_known_addr {
host: "127.9.157.188"
port: 44451
}
attrs {
promote: true
}
}
dest_uuid: "41918f3984d74e9a9e03e35f07467cb1"
cas_config_opid_index: 5
I20260811 17:19:42.199522 20058 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: NON_VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: true } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: NON_VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: true } }
I20260811 17:19:42.201084 20899 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER bac5ec8af44844e388ff31729b5549bc (127.9.157.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: NON_VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: true } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: NON_VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: true } } }
I20260811 17:19:42.201546 21096 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:42.202556 20899 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.157.188:44451
I20260811 17:19:42.202811 20899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "41918f3984d74e9a9e03e35f07467cb1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: NON_VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: true } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: NON_VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: true } } }
I20260811 17:19:42.202929 20899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:42.203070 21096 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 1 FOLLOWER]: Refusing update from remote peer 41918f3984d74e9a9e03e35f07467cb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 17:19:42.203918 20898 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: NON_VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:42.205018 20905 master.cc:537] Master@127.9.157.188:44451 shutting down...
I20260811 17:19:42.208652 21096 raft_consensus.cc:858] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 44451 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: NON_VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: NON_VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: true } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: NON_VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: true } }
W20260811 17:19:42.208801 21096 raft_consensus.cc:1565] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 44451 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: NON_VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: NON_VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: true } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: NON_VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: true } }
I20260811 17:19:42.210455 21096 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 2409126f018b4ea38b67722ebcfdc44d (127.9.157.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: NON_VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: true } } }
I20260811 17:19:42.210896 21146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.203615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48674 (local address 127.9.157.189:33385)
0811 17:19:42.205700 (+ 2085us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:42.205708 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:42.205730 (+ 22us) server_negotiation.cc:407] Connection header received
0811 17:19:42.205798 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:42.205803 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:42.205864 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:42.206013 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:42.206394 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.207202 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.209156 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.209939 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.210041 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.210216 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:42.210657 (+ 441us) server_negotiation.cc:299] Negotiation successful
0811 17:19:42.210753 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1943}
I20260811 17:19:42.211144 20895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.203308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:33385 (local address 127.0.0.1:48674)
0811 17:19:42.204148 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:42.204167 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:42.204280 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:42.206043 (+ 1763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:42.206047 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:42.206068 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:42.206197 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.206207 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.207393 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.207398 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.208753 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.208992 (+ 239us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.210354 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.210361 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.210475 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.210534 (+ 59us) client_negotiation.cc:769] Sending connection context
0811 17:19:42.210620 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 17:19:42.210725 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 17:19:42.211891 20972 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [term 1 LEARNER]: Refusing update from remote peer 41918f3984d74e9a9e03e35f07467cb1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 17:19:42.212462 20899 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: NON_VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 17:19:42.214672 20972 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER bac5ec8af44844e388ff31729b5549bc (127.9.157.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: NON_VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: true } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: NON_VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: true } } }
I20260811 17:19:42.217123 21013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41918f3984d74e9a9e03e35f07467cb1. Latest consensus state: current_term: 1 leader_uuid: "41918f3984d74e9a9e03e35f07467cb1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: NON_VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: true } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: NON_VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: true } } }
I20260811 17:19:42.217281 21013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [sys.catalog]: This master's current role is: LEARNER
I20260811 17:19:42.217411 21013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "41918f3984d74e9a9e03e35f07467cb1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: NON_VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: true } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: NON_VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: true } } }
I20260811 17:19:42.217502 21013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [sys.catalog]: This master's current role is: LEARNER
I20260811 17:19:42.218535 20899 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1: attempting to promote NON_VOTER 2409126f018b4ea38b67722ebcfdc44d to VOTER
I20260811 17:19:42.218681 21148 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "41918f3984d74e9a9e03e35f07467cb1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: NON_VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: true } } }
I20260811 17:19:42.218818 21148 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 17:19:42.218937 21148 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "41918f3984d74e9a9e03e35f07467cb1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: NON_VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: true } } }
I20260811 17:19:42.219007 21148 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 17:19:42.219103 20899 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: NON_VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: true } }
I20260811 17:19:42.219954 21189 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41918f3984d74e9a9e03e35f07467cb1. Latest consensus state: current_term: 1 leader_uuid: "41918f3984d74e9a9e03e35f07467cb1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: NON_VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: true } } }
I20260811 17:19:42.220118 21189 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 17:19:42.220837 20972 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [term 1 LEARNER]: Refusing update from remote peer 41918f3984d74e9a9e03e35f07467cb1: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 17:19:42.221446 20898 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260811 17:19:42.222035 20035 consensus_peers.cc:487] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 -> Peer bac5ec8af44844e388ff31729b5549bc (127.9.157.188:44451): Couldn't send request to peer bac5ec8af44844e388ff31729b5549bc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 17:19:42.223379 20898 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 2409126f018b4ea38b67722ebcfdc44d (127.9.157.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: NON_VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: true } } }
I20260811 17:19:42.224884 20898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "41918f3984d74e9a9e03e35f07467cb1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: NON_VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: true } } }
I20260811 17:19:42.225016 20898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:42.225191 20898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "41918f3984d74e9a9e03e35f07467cb1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: NON_VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: true } } }
I20260811 17:19:42.225310 20898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:42.226251 20972 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 2409126f018b4ea38b67722ebcfdc44d (127.9.157.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: NON_VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: true } } }
I20260811 17:19:42.227838 21194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "41918f3984d74e9a9e03e35f07467cb1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: NON_VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: true } } }
I20260811 17:19:42.227998 21194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:19:42.229614 21184 mvcc.cc:205] Tried to move back new op lower bound from 7317376125843722240 to 7317376125565739008. Current Snapshot: MvccSnapshot[applied={T|T < 7317376125601636352}]
I20260811 17:19:42.229888 20905 raft_consensus.cc:2229] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260811 17:19:42.230165 20905 raft_consensus.cc:2258] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260811 17:19:42.230278 20905 tablet_replica.cc:330] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc: stopping tablet replica
I20260811 17:19:42.238294 20905 master.cc:552] Master@127.9.157.188:44451 shutdown complete.
I20260811 17:19:42.238387 20905 master_runner.cc:305] Clearing existing system tablet
I20260811 17:19:42.238682 20905 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:42.238760 20905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:42.244099 20905 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260811 17:19:42.245108 21202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:42.245409 20905 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:42.245569 20905 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 17:19:42.245609 20905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-2/wal
uuid: "bac5ec8af44844e388ff31729b5549bc"
format_stamp: "Formatted at 2026-08-11 17:19:42 on dist-test-slave-mjjr"
server_key: "0ede314c7c1d4ec35b1b99d7c71f3e88"
server_key_iv: "c64efe91831abc501f9d2d292ab75a3f"
server_key_version: "encryptionkey@0"
I20260811 17:19:42.247284 20539 heartbeater.cc:503] Master 127.9.157.190:34565 was elected leader, sending a full tablet report...
I20260811 17:19:42.247587 20905 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:42.249970 20905 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 17:19:42.250092 20905 log.cc:1196] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 17:19:42.250386 20905 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc: Deleting consensus metadata
I20260811 17:19:42.250579 20905 master_runner.cc:315] Copying system tablet from 127.9.157.190:34565
I20260811 17:19:42.251879 20905 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc: tablet copy: Beginning tablet copy session from remote peer at address 127.9.157.190:34565
I20260811 17:19:42.260838 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.252544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53072 (local address 127.9.157.190:34565)
0811 17:19:42.252684 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:42.252690 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:42.252921 (+ 231us) server_negotiation.cc:407] Connection header received
0811 17:19:42.253073 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:42.253076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:42.253131 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:42.253220 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:42.253747 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.254345 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.255343 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.255748 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.255888 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.257390 (+ 1502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:42.257394 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:42.257396 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:42.257431 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:42.259020 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.259487 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.259491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.259493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.259583 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.259915 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.259919 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.259920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.260131 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:42.260244 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:42.260528 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 17:19:42.260650 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 17:19:42.260859 21213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.252479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.0.0.1:53072)
0811 17:19:42.252818 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:42.252831 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:42.252971 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:42.253241 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:42.253244 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:42.253466 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:42.253625 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.253632 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.254466 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.254471 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.255137 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.255148 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.255883 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.255887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.255987 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.256664 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:42.256689 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:42.257240 (+ 551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:42.259145 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.259150 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.259153 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.259380 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.259708 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.259711 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.259713 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.259807 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.260235 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:42.260238 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:42.260355 (+ 117us) client_negotiation.cc:769] Sending connection context
0811 17:19:42.260469 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 17:19:42.260643 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":91,"threads_started":1}
I20260811 17:19:42.261849 20084 tablet_copy_service.cc:140] P 41918f3984d74e9a9e03e35f07467cb1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer bac5ec8af44844e388ff31729b5549bc ({username='oryx', principal='oryx/127.9.157.188@KRBTEST.COM'} at 127.0.0.1:53072)
I20260811 17:19:42.261981 20084 tablet_copy_service.cc:161] P 41918f3984d74e9a9e03e35f07467cb1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer bac5ec8af44844e388ff31729b5549bc at {username='oryx', principal='oryx/127.9.157.188@KRBTEST.COM'} at 127.0.0.1:53072: session id = bac5ec8af44844e388ff31729b5549bc-00000000000000000000000000000000
I20260811 17:19:42.262866 20084 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 17:19:42.264276 20905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 17:19:42.266459 20905 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc: tablet copy: Starting download of 0 data blocks...
I20260811 17:19:42.266705 20905 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc: tablet copy: Starting download of 1 WAL segments...
I20260811 17:19:42.268523 20905 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 17:19:42.270421 20084 tablet_copy_service.cc:342] P 41918f3984d74e9a9e03e35f07467cb1: Request end of tablet copy session bac5ec8af44844e388ff31729b5549bc-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.157.188@KRBTEST.COM'} at 127.0.0.1:53072
I20260811 17:19:42.270543 20084 tablet_copy_service.cc:433] P 41918f3984d74e9a9e03e35f07467cb1: ending tablet copy session bac5ec8af44844e388ff31729b5549bc-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer bac5ec8af44844e388ff31729b5549bc
I20260811 17:19:42.278347 20905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:42.280192 21217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:42.280589 21219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:42.280727 21222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:42.280948 20905 server_base.cc:1034] running on GCE node
I20260811 17:19:42.281213 20905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:42.281610 20905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:42.282752 20905 hybrid_clock.cc:648] HybridClock initialized: now 1786468782282740 us; error 29 us; skew 500 ppm
Aug 11 17:19:42 dist-test-slave-mjjr krb5kdc[19981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468782, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:42.284385 20905 init.cc:377] Logged in from keytab as oryx/127.9.157.188@KRBTEST.COM (short username oryx)
I20260811 17:19:42.284929 20905 webserver.cc:460] Webserver started at http://127.9.157.188:43535/ using document root <none> and password file <none>
I20260811 17:19:42.285140 20905 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:42.285197 20905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:42.286573 20905 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 17:19:42.287742 21235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:42.287992 20905 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 17:19:42.288095 20905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-2/wal
uuid: "bac5ec8af44844e388ff31729b5549bc"
format_stamp: "Formatted at 2026-08-11 17:19:42 on dist-test-slave-mjjr"
server_key: "0ede314c7c1d4ec35b1b99d7c71f3e88"
server_key_iv: "c64efe91831abc501f9d2d292ab75a3f"
server_key_version: "encryptionkey@0"
I20260811 17:19:42.288210 20905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:42.300293 20905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:42.302542 20905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:42.308040 20905 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.188:44451
I20260811 17:19:42.308776 20905 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/master-2/data/info.pb
I20260811 17:19:42.309214 21311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.188:44451 every 8 connection(s)
I20260811 17:19:42.310506 21312 sys_catalog.cc:263] Verifying existing consensus state
I20260811 17:19:42.311424 21312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc: Bootstrap starting.
I20260811 17:19:42.318425 21312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 17:19:42.318807 21312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc: Bootstrap complete.
I20260811 17:19:42.319386 21312 raft_consensus.cc:348] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: NON_VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: true } }
I20260811 17:19:42.319528 21312 raft_consensus.cc:729] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bac5ec8af44844e388ff31729b5549bc, State: Initialized, Role: LEARNER
I20260811 17:19:42.319675 21312 consensus_queue.cc:260] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: NON_VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: true } }
I20260811 17:19:42.320080 21315 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: NON_VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: true } } }
I20260811 17:19:42.320261 21315 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [sys.catalog]: This master's current role is: LEARNER
I20260811 17:19:42.320258 21312 sys_catalog.cc:564] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:19:42.324612 21337 catalog_manager.cc:1259] Loaded cluster ID: 7725b3cbf87546e186272ad03fb69783
I20260811 17:19:42.324707 21337 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc: loading cluster ID for follower catalog manager: success
I20260811 17:19:42.326345 21337 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc: acquiring CA information for follower catalog manager: success
I20260811 17:19:42.327555 21337 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 17:19:42.358050 21182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:42.358493 21182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:42.358573 21182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:42.364712 21182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:42.364816 21182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:42.364907 21182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:19:42.373121 21182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:39005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.129.keytab
--principal=oryx/127.9.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:38203
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=33591
--tserver_master_addrs=127.9.157.190:34565,127.9.157.189:33385,127.9.157.188:44451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:42.374642 21182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:42.375963 21182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:42.387830 21348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:42.388247 21345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:42.387822 21344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:42.387988 21182 server_base.cc:1034] running on GCE node
I20260811 17:19:42.388896 21182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:42.389734 21182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:42.390928 21182 hybrid_clock.cc:648] HybridClock initialized: now 1786468782390901 us; error 43 us; skew 500 ppm
Aug 11 17:19:42 dist-test-slave-mjjr krb5kdc[19981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468782, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:42.393842 21182 init.cc:377] Logged in from keytab as oryx/127.9.157.129@KRBTEST.COM (short username oryx)
I20260811 17:19:42.394856 21182 webserver.cc:460] Webserver started at http://127.9.157.129:33591/ using document root <none> and password file <none>
I20260811 17:19:42.395462 21182 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:42.395565 21182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:42.400413 21182 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:42.402681 21359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:42.404090 21182 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:42.404240 21182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "6756578bcb7b438b8a5bcbaaecc2914a"
format_stamp: "Formatted at 2026-08-11 17:19:40 on dist-test-slave-mjjr"
server_key: "29ed5ac84a1ddb486e50e95f70ab7c96"
server_key_iv: "224f0f05bece7085e7856ab756c278a4"
server_key_version: "encryptionkey@0"
I20260811 17:19:42.404834 21182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:42.422173 21182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:42.425940 21182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:42.426165 21182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:42.426805 21182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:42.427750 21182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:42.427799 21182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:42.427843 21182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:42.427870 21182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:42.438057 21182 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:38203
I20260811 17:19:42.438082 21528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:38203 every 8 connection(s)
I20260811 17:19:42.439389 21182 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:19:42.444597 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 21182
I20260811 17:19:42.444818 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 20344
Aug 11 17:19:42 dist-test-slave-mjjr krb5kdc[19981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468782, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.188@KRBTEST.COM
I20260811 17:19:42.475016 21534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.441482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:55009 (local address 127.9.157.188:44451)
0811 17:19:42.442172 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:42.442190 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:42.442889 (+ 699us) server_negotiation.cc:407] Connection header received
0811 17:19:42.444012 (+ 1123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:42.444015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:42.444070 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:42.444136 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:42.457049 (+ 12913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.458136 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.459983 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.460355 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.460445 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.470755 (+ 10310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:42.470759 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:42.470761 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:42.470791 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:42.472460 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.473147 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.473150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.473152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.473237 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.473723 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.473728 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.473730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.473957 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:42.474027 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:42.474645 (+ 618us) server_negotiation.cc:299] Negotiation successful
0811 17:19:42.474803 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":94,"threads_started":1}
I20260811 17:19:42.476161 21536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.441958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:44451 (local address 127.9.157.129:55009)
0811 17:19:42.442455 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:42.442510 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:42.443731 (+ 1221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:42.444361 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:42.444372 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:42.455275 (+ 10903us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:42.456778 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.456813 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.458367 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.458373 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.459811 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.459828 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.460690 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.460695 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.460944 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.461651 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:42.461677 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:42.470503 (+ 8826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:42.472630 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.472638 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.472657 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.473009 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.473350 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.473353 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.473355 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.473593 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.474092 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:42.474097 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:42.474378 (+ 281us) client_negotiation.cc:769] Sending connection context
0811 17:19:42.474580 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 17:19:42.474961 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":142,"threads_started":1}
I20260811 17:19:42.478125 21529 heartbeater.cc:347] Connected to a master server at 127.9.157.188:44451
I20260811 17:19:42.478448 21529 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:42.479104 21529 heartbeater.cc:511] Master 127.9.157.188:44451 requested a full tablet report, sending...
I20260811 17:19:42.481355 21255 ts_manager.cc:194] Registered new tserver with Master: 6756578bcb7b438b8a5bcbaaecc2914a (127.9.157.129:38203)
Aug 11 17:19:42 dist-test-slave-mjjr krb5kdc[19981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468782, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.189@KRBTEST.COM
Aug 11 17:19:42 dist-test-slave-mjjr krb5kdc[19981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468782, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM
I20260811 17:19:42.486014 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:37833
--local_ip_for_outbound_sockets=127.9.157.130
--tserver_master_addrs=127.9.157.190:34565
--webserver_port=38247
--webserver_interface=127.9.157.130
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.130.keytab
--principal=oryx/127.9.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:39005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.157.190:34565,127.9.157.189:33385
--tserver_master_addrs=127.9.157.190:34565,127.9.157.189:33385,127.9.157.188:44451 with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:19:42.486925 20743 heartbeater.cc:503] Master 127.9.157.190:34565 was elected leader, sending a full tablet report...
I20260811 17:19:42.488449 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.443222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:53839 (local address 127.9.157.190:34565)
0811 17:19:42.443357 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:42.443360 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:42.443457 (+ 97us) server_negotiation.cc:407] Connection header received
0811 17:19:42.444089 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:42.444113 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:42.444174 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:42.444253 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:42.478914 (+ 34661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.479697 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.480854 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.481184 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.481265 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.484602 (+ 3337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:42.484609 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:42.484615 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:42.484657 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:42.486320 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.486876 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.486879 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.486881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.486967 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.487313 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.487317 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.487318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.487577 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:42.487706 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:42.488142 (+ 436us) server_negotiation.cc:299] Negotiation successful
0811 17:19:42.488281 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 17:19:42.489074 21543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.443052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.9.157.129:53839)
0811 17:19:42.443359 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:42.443373 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:42.443732 (+ 359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:42.444379 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:42.444381 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:42.478581 (+ 34200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:42.478730 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.478737 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.479856 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.479859 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.480714 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.480722 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.481433 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.481436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.481532 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.482410 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:42.482425 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:42.484410 (+ 1985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:42.486469 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.486476 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.486480 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.486738 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.487066 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.487071 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.487073 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.487160 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.487770 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:42.487773 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:42.487874 (+ 101us) client_negotiation.cc:769] Sending connection context
0811 17:19:42.488683 (+ 809us) client_negotiation.cc:240] Negotiation successful
0811 17:19:42.488852 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":112,"threads_started":1}
I20260811 17:19:42.489215 21146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.441463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:41929 (local address 127.9.157.189:33385)
0811 17:19:42.441601 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:42.441606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:42.442625 (+ 1019us) server_negotiation.cc:407] Connection header received
0811 17:19:42.443966 (+ 1341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:42.443973 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:42.444041 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:42.444131 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:42.478528 (+ 34397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.479365 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.480722 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.481058 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.481139 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.484346 (+ 3207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:42.484354 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:42.484359 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:42.484393 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:42.486318 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.487514 (+ 1196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.487521 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.487526 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.487734 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.488178 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.488182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.488183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.488488 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:42.488595 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:42.488931 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 17:19:42.489054 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 17:19:42.489602 21541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.441958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:33385 (local address 127.9.157.129:41929)
0811 17:19:42.442454 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:42.442510 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:42.443732 (+ 1222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:42.444361 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:42.444371 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:42.478220 (+ 33849us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:42.478369 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.478376 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.479505 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.479509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.480564 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.480575 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.481323 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.481327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.481451 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.482241 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:42.482263 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:42.484131 (+ 1868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:42.486477 (+ 2346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.486482 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.486488 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.487380 (+ 892us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.487888 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.487893 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.487898 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.488048 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.488655 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:42.488658 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:42.488741 (+ 83us) client_negotiation.cc:769] Sending connection context
0811 17:19:42.488858 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 17:19:42.489013 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":59,"thread_start_us":182,"threads_started":1}
I20260811 17:19:42.489925 21531 heartbeater.cc:347] Connected to a master server at 127.9.157.190:34565
I20260811 17:19:42.490032 21531 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:42.490257 21531 heartbeater.cc:511] Master 127.9.157.190:34565 requested a full tablet report, sending...
I20260811 17:19:42.490293 21532 heartbeater.cc:347] Connected to a master server at 127.9.157.189:33385
I20260811 17:19:42.490379 21532 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:42.490520 21532 heartbeater.cc:511] Master 127.9.157.189:33385 requested a full tablet report, sending...
I20260811 17:19:42.491163 20058 ts_manager.cc:194] Re-registered known tserver with Master: 6756578bcb7b438b8a5bcbaaecc2914a (127.9.157.129:38203)
I20260811 17:19:42.491825 20960 ts_manager.cc:194] Registered new tserver with Master: 6756578bcb7b438b8a5bcbaaecc2914a (127.9.157.129:38203)
I20260811 17:19:42.491948 20058 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.157.129@KRBTEST.COM'} at 127.9.157.129:53839
W20260811 17:19:42.616065 21546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:42.616446 21546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:42.616526 21546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:42.621477 21546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:42.621614 21546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:42.621718 21546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:19:42.627547 21546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:39005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.130.keytab
--principal=oryx/127.9.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:37833
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=38247
--tserver_master_addrs=127.9.157.190:34565,127.9.157.189:33385,127.9.157.188:44451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:42.629068 21546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:42.630556 21546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:42.643865 21556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:42.643898 21552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:42.643885 21553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:42.644050 21546 server_base.cc:1034] running on GCE node
I20260811 17:19:42.644657 21546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:42.645414 21546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:42.646621 21546 hybrid_clock.cc:648] HybridClock initialized: now 1786468782646589 us; error 46 us; skew 500 ppm
Aug 11 17:19:42 dist-test-slave-mjjr krb5kdc[19981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468782, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:42.650142 21546 init.cc:377] Logged in from keytab as oryx/127.9.157.130@KRBTEST.COM (short username oryx)
I20260811 17:19:42.651180 21546 webserver.cc:460] Webserver started at http://127.9.157.130:38247/ using document root <none> and password file <none>
I20260811 17:19:42.651885 21546 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:42.652016 21546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:42.657074 21546 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 17:19:42.659492 21570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:42.661026 21546 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:19:42.661288 21546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "7432ee13358d444494d13841c91bfd5f"
format_stamp: "Formatted at 2026-08-11 17:19:41 on dist-test-slave-mjjr"
server_key: "e1aa114d69bfd56287c34baccecaaec4"
server_key_iv: "198d0e0334ea7c1c291063294891ae22"
server_key_version: "encryptionkey@0"
I20260811 17:19:42.661916 21546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:42.685774 21546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:42.689837 21546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:42.690078 21546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:42.690773 21546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:42.691817 21546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:42.691887 21546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:42.691957 21546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:42.692008 21546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:42.702961 21546 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:37833
I20260811 17:19:42.703011 21746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:37833 every 8 connection(s)
I20260811 17:19:42.704154 21546 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:19:42.709002 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 21546
I20260811 17:19:42.709221 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 20547
Aug 11 17:19:42 dist-test-slave-mjjr krb5kdc[19981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468782, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM
I20260811 17:19:42.724118 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.706302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:46095 (local address 127.9.157.190:34565)
0811 17:19:42.706542 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:42.706547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:42.707700 (+ 1153us) server_negotiation.cc:407] Connection header received
0811 17:19:42.709310 (+ 1610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:42.709314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:42.709379 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:42.709434 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:42.711424 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.712194 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.713965 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.714316 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.714409 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.718163 (+ 3754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:42.718170 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:42.718174 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:42.718213 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:42.720917 (+ 2704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.722034 (+ 1117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.722040 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.722044 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.722160 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.722615 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.722620 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.722624 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.722880 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:42.722960 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:42.723715 (+ 755us) server_negotiation.cc:299] Negotiation successful
0811 17:19:42.723877 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
Aug 11 17:19:42 dist-test-slave-mjjr krb5kdc[19981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468782, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.189@KRBTEST.COM
I20260811 17:19:42.726227 21756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.706899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.9.157.130:46095)
0811 17:19:42.707483 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:42.707542 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:42.708876 (+ 1334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:42.709729 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:42.709741 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:42.710391 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:42.711227 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.711250 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.712401 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.712407 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.713798 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.713812 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.714652 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.714658 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.714906 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.715694 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:42.715720 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:42.717898 (+ 2178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:42.721089 (+ 3191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.721099 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.721177 (+ 78us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.721902 (+ 725us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.722289 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.722294 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.722299 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.722510 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.723029 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:42.723039 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:42.723416 (+ 377us) client_negotiation.cc:769] Sending connection context
0811 17:19:42.724639 (+ 1223us) client_negotiation.cc:240] Negotiation successful
0811 17:19:42.725062 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":57,"spinlock_wait_cycles":896,"thread_start_us":150,"threads_started":1}
I20260811 17:19:42.728199 21752 heartbeater.cc:347] Connected to a master server at 127.9.157.190:34565
I20260811 17:19:42.728436 21752 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:42.729084 21752 heartbeater.cc:511] Master 127.9.157.190:34565 requested a full tablet report, sending...
I20260811 17:19:42.736238 20058 ts_manager.cc:194] Re-registered known tserver with Master: 7432ee13358d444494d13841c91bfd5f (127.9.157.130:37833)
I20260811 17:19:42.736914 20058 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.157.130@KRBTEST.COM'} at 127.9.157.130:46095
Aug 11 17:19:42 dist-test-slave-mjjr krb5kdc[19981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468782, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.188@KRBTEST.COM
I20260811 17:19:42.753108 21146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.706360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:45331 (local address 127.9.157.189:33385)
0811 17:19:42.706543 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:42.706550 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:42.707708 (+ 1158us) server_negotiation.cc:407] Connection header received
0811 17:19:42.709148 (+ 1440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:42.709152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:42.709219 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:42.709283 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:42.711463 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.712758 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.714432 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.720579 (+ 6147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.720754 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.727035 (+ 6281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:42.727043 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:42.727050 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:42.727097 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:42.750491 (+ 23394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.751159 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.751166 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.751170 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.751325 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.751712 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.751718 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.751723 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.752007 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:42.752093 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:42.752681 (+ 588us) server_negotiation.cc:299] Negotiation successful
0811 17:19:42.752842 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 17:19:42.753508 21534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.708271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:47445 (local address 127.9.157.188:44451)
0811 17:19:42.708427 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:42.708432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:42.708530 (+ 98us) server_negotiation.cc:407] Connection header received
0811 17:19:42.709129 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:42.709135 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:42.709198 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:42.709252 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:42.743350 (+ 34098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.744064 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.745309 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.745895 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.746080 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.748845 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:42.748852 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:42.748857 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:42.748895 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:42.751632 (+ 2737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.752157 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.752161 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.752162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.752249 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.752580 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.752584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.752585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.752782 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:42.752848 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:42.753192 (+ 344us) server_negotiation.cc:299] Negotiation successful
0811 17:19:42.753350 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 17:19:42.754105 21758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.708115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:44451 (local address 127.9.157.130:47445)
0811 17:19:42.708425 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:42.708441 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:42.708861 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:42.742800 (+ 33939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:42.742803 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:42.743018 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:42.743153 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.743159 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.744231 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.744237 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.745136 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.745151 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.746082 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.746088 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.746218 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.746939 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:42.746958 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:42.748632 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:42.751789 (+ 3157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.751793 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.751795 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.752043 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.752381 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.752384 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.752386 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.752477 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.752921 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:42.752923 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:42.753012 (+ 89us) client_negotiation.cc:769] Sending connection context
0811 17:19:42.753685 (+ 673us) client_negotiation.cc:240] Negotiation successful
0811 17:19:42.753913 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":120,"threads_started":1}
I20260811 17:19:42.755177 21747 heartbeater.cc:347] Connected to a master server at 127.9.157.188:44451
I20260811 17:19:42.755199 21760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.735249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33730 (local address 127.9.157.188:44451)
0811 17:19:42.749966 (+ 14717us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:42.749970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:42.749982 (+ 12us) server_negotiation.cc:407] Connection header received
0811 17:19:42.750024 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:42.750026 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:42.750083 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:42.750234 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:42.750611 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.751789 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.753350 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.754074 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.754201 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.754394 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:42.754966 (+ 572us) server_negotiation.cc:299] Negotiation successful
0811 17:19:42.755058 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14628,"thread_start_us":409,"threads_started":1}
I20260811 17:19:42.755321 21747 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:42.755518 21759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.735186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:44451 (local address 127.0.0.1:33730)
0811 17:19:42.735857 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:42.735873 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:42.735992 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:42.750269 (+ 14277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:42.750273 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:42.750304 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:42.750446 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.750456 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.751932 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.751937 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.753205 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.753220 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.754615 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.754622 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.754761 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.754807 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 17:19:42.754895 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 17:19:42.755049 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":457,"threads_started":1}
I20260811 17:19:42.755584 21747 heartbeater.cc:511] Master 127.9.157.188:44451 requested a full tablet report, sending...
I20260811 17:19:42.756028 21757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.706899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:33385 (local address 127.9.157.130:45331)
0811 17:19:42.707484 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:42.707542 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:42.708855 (+ 1313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:42.709730 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:42.709741 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:42.710409 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:42.711231 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.711254 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.712978 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.712984 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.714248 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.714265 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.721061 (+ 6796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.721068 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.721258 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.724589 (+ 3331us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:42.724612 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:42.726758 (+ 2146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:42.750651 (+ 23893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.750658 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.750663 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.751017 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.751462 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.751468 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.751472 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.751573 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.752176 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:42.752182 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:42.752305 (+ 123us) client_negotiation.cc:769] Sending connection context
0811 17:19:42.753867 (+ 1562us) client_negotiation.cc:240] Negotiation successful
0811 17:19:42.754082 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":92,"spinlock_wait_cycles":27648,"thread_start_us":155,"threads_started":1}
I20260811 17:19:42.756386 21269 raft_consensus.cc:1203] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 17:19:42.756695 21753 heartbeater.cc:347] Connected to a master server at 127.9.157.189:33385
I20260811 17:19:42.756783 21753 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:42.756956 21753 heartbeater.cc:511] Master 127.9.157.189:33385 requested a full tablet report, sending...
I20260811 17:19:42.756875 21315 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41918f3984d74e9a9e03e35f07467cb1. Latest consensus state: current_term: 1 leader_uuid: "41918f3984d74e9a9e03e35f07467cb1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: NON_VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: true } } }
I20260811 17:19:42.757050 21315 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [sys.catalog]: This master's current role is: LEARNER
I20260811 17:19:42.757395 21255 ts_manager.cc:194] Registered new tserver with Master: 7432ee13358d444494d13841c91bfd5f (127.9.157.130:37833)
I20260811 17:19:42.757707 20960 ts_manager.cc:194] Registered new tserver with Master: 7432ee13358d444494d13841c91bfd5f (127.9.157.130:37833)
I20260811 17:19:42.759145 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:37867
--local_ip_for_outbound_sockets=127.9.157.131
--tserver_master_addrs=127.9.157.190:34565
--webserver_port=36981
--webserver_interface=127.9.157.131
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.131.keytab
--principal=oryx/127.9.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:39005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.157.190:34565,127.9.157.189:33385
--tserver_master_addrs=127.9.157.190:34565,127.9.157.189:33385,127.9.157.188:44451 with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:42.885752 21761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:42.886117 21761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:42.886198 21761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:42.890602 21761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:42.890712 21761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:42.890810 21761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:19:42.896376 21761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:39005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.131.keytab
--principal=oryx/127.9.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:37867
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=36981
--tserver_master_addrs=127.9.157.190:34565,127.9.157.189:33385,127.9.157.188:44451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:42.897716 21761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:42.898909 21761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:42.909744 21770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:42.909742 21766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:42.909742 21768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:42.909973 21761 server_base.cc:1034] running on GCE node
I20260811 17:19:42.910727 21761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:42.911475 21761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:42.912681 21761 hybrid_clock.cc:648] HybridClock initialized: now 1786468782912656 us; error 38 us; skew 500 ppm
Aug 11 17:19:42 dist-test-slave-mjjr krb5kdc[19981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468782, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:42.915850 21761 init.cc:377] Logged in from keytab as oryx/127.9.157.131@KRBTEST.COM (short username oryx)
I20260811 17:19:42.916915 21761 webserver.cc:460] Webserver started at http://127.9.157.131:36981/ using document root <none> and password file <none>
I20260811 17:19:42.917572 21761 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:42.917701 21761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:42.922264 21761 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 17:19:42.924813 21777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:42.926194 21761 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:42.926376 21761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "4cd592e9c42a4fe8881afa489954dd8e"
format_stamp: "Formatted at 2026-08-11 17:19:41 on dist-test-slave-mjjr"
server_key: "c1f6350bd9b592680116d27a25a970d6"
server_key_iv: "9e755240c94edb9ed6bab427488b4d33"
server_key_version: "encryptionkey@0"
I20260811 17:19:42.927006 21761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:42.941813 21761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:42.945530 21761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:42.945767 21761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:42.946517 21761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:42.947523 21761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:42.947597 21761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:42.947695 21761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:42.947752 21761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:42.959314 21761 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:37867
I20260811 17:19:42.959403 21944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:37867 every 8 connection(s)
I20260811 17:19:42.960659 21761 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:19:42.967587 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 21761
Aug 11 17:19:42 dist-test-slave-mjjr krb5kdc[19981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468782, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.188@KRBTEST.COM
Aug 11 17:19:42 dist-test-slave-mjjr krb5kdc[19981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468782, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.189@KRBTEST.COM
Aug 11 17:19:42 dist-test-slave-mjjr krb5kdc[19981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468782, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM
I20260811 17:19:42.983007 21146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.962777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:51845 (local address 127.9.157.189:33385)
0811 17:19:42.962982 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:42.962989 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:42.964053 (+ 1064us) server_negotiation.cc:407] Connection header received
0811 17:19:42.965496 (+ 1443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:42.965499 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:42.965552 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:42.965616 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:42.967422 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.968106 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.969576 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.969921 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.970032 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.974769 (+ 4737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:42.974773 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:42.974777 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:42.974803 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:42.977252 (+ 2449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.979962 (+ 2710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.979965 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.979967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.980068 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.981254 (+ 1186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.981259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.981262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.981808 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:42.981907 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:42.982606 (+ 699us) server_negotiation.cc:299] Negotiation successful
0811 17:19:42.982755 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 17:19:42.984709 21953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.963300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:33385 (local address 127.9.157.131:51845)
0811 17:19:42.963870 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:42.963930 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:42.965214 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:42.965903 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:42.965911 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:42.966484 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:42.967255 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.967272 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.968279 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.968282 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.969428 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.969439 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.970235 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.970259 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.970509 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.971810 (+ 1301us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:42.971826 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:42.974497 (+ 2671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:42.979341 (+ 4844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.979351 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.979376 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.979825 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.980174 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.980180 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.980184 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.980418 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.981974 (+ 1556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:42.981977 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:42.982183 (+ 206us) client_negotiation.cc:769] Sending connection context
0811 17:19:42.982471 (+ 288us) client_negotiation.cc:240] Negotiation successful
0811 17:19:42.982947 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":219,"spinlock_wait_cycles":58368,"thread_start_us":181,"threads_started":1}
I20260811 17:19:42.984972 21760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.964651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:54859 (local address 127.9.157.188:44451)
0811 17:19:42.964786 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:42.964791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:42.965033 (+ 242us) server_negotiation.cc:407] Connection header received
0811 17:19:42.965634 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:42.965639 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:42.965701 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:42.965813 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:42.967476 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.968396 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.969749 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.970134 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.970217 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.974733 (+ 4516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:42.974738 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:42.974740 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:42.974773 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:42.978084 (+ 3311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.982312 (+ 4228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.982317 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.982319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.982433 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.983852 (+ 1419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.983857 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.983859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.984110 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:42.984189 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:42.984594 (+ 405us) server_negotiation.cc:299] Negotiation successful
0811 17:19:42.984812 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 17:19:42.985819 21960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.970061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48680 (local address 127.9.157.189:33385)
0811 17:19:42.971117 (+ 1056us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:42.971122 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:42.972827 (+ 1705us) server_negotiation.cc:407] Connection header received
0811 17:19:42.973507 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:42.973511 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:42.973573 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:42.973721 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:42.977115 (+ 3394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.978001 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.979497 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.979891 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.979970 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.981797 (+ 1827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:42.981801 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:42.981804 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:42.981824 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:42.983583 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.984184 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.984188 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.984189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.984277 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.984612 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.984615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.984616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.984992 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:42.985108 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:42.985531 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 17:19:42.985663 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":964,"thread_start_us":94,"threads_started":1}
I20260811 17:19:42.986159 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.962768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:40865 (local address 127.9.157.190:34565)
0811 17:19:42.962911 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:42.962916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:42.964056 (+ 1140us) server_negotiation.cc:407] Connection header received
0811 17:19:42.965470 (+ 1414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:42.965474 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:42.965541 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:42.965632 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:42.969968 (+ 4336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.970827 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.971898 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.972250 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.972337 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.975236 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:42.975241 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:42.975245 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:42.975277 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:42.979190 (+ 3913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.980675 (+ 1485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.980680 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.980682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.980793 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.981159 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.981163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.981165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.981431 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:42.981534 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:42.985814 (+ 4280us) server_negotiation.cc:299] Negotiation successful
0811 17:19:42.985969 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 17:19:42.986831 21534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.972900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33734 (local address 127.9.157.188:44451)
0811 17:19:42.973026 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:42.973031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:42.976686 (+ 3655us) server_negotiation.cc:407] Connection header received
0811 17:19:42.976887 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:42.976891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:42.976946 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:42.977001 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:42.978077 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.978880 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.980480 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.980883 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.980965 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.982757 (+ 1792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:42.982761 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:42.982764 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:42.982794 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:42.984696 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.985271 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.985277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.985281 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.985381 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.985757 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.985761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.985763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.985997 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:42.986108 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:42.986550 (+ 442us) server_negotiation.cc:299] Negotiation successful
0811 17:19:42.986679 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 17:19:42.986789 21948 heartbeater.cc:347] Connected to a master server at 127.9.157.189:33385
I20260811 17:19:42.987079 21948 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:42.987190 21951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.963282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.9.157.131:40865)
0811 17:19:42.963870 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:42.963930 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:42.965214 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:42.965907 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:42.965913 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:42.966484 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:42.969800 (+ 3316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.969810 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.970999 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.971003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.971716 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.971740 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.972540 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.972544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.972647 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.973261 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:42.973277 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:42.975018 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:42.980230 (+ 5212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.980235 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.980238 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.980542 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.980926 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.980932 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.980936 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.981036 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.981645 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:42.981656 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:42.985528 (+ 3872us) client_negotiation.cc:769] Sending connection context
0811 17:19:42.986320 (+ 792us) client_negotiation.cc:240] Negotiation successful
0811 17:19:42.986547 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":147,"spinlock_wait_cycles":54016,"thread_start_us":143,"threads_started":1}
I20260811 17:19:42.987743 21948 heartbeater.cc:511] Master 127.9.157.189:33385 requested a full tablet report, sending...
I20260811 17:19:42.988409 21947 heartbeater.cc:347] Connected to a master server at 127.9.157.190:34565
I20260811 17:19:42.988500 21947 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:42.988683 21954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.964471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:44451 (local address 127.9.157.131:54859)
0811 17:19:42.964832 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:42.964849 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:42.965218 (+ 369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:42.965944 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:42.965947 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:42.966487 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:42.967258 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.967276 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.968577 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.968584 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.969574 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:42.969585 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.970416 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.970420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:42.970538 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.971371 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:42.971396 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:42.974497 (+ 3101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:42.980819 (+ 6322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.980823 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.980824 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.981099 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.983585 (+ 2486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:42.983589 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:42.983591 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:42.983721 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:42.984273 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:42.984276 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:42.984393 (+ 117us) client_negotiation.cc:769] Sending connection context
0811 17:19:42.988233 (+ 3840us) client_negotiation.cc:240] Negotiation successful
0811 17:19:42.988419 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":128,"spinlock_wait_cycles":16000,"thread_start_us":129,"threads_started":1}
I20260811 17:19:42.988700 21947 heartbeater.cc:511] Master 127.9.157.190:34565 requested a full tablet report, sending...
I20260811 17:19:42.989483 21946 heartbeater.cc:347] Connected to a master server at 127.9.157.188:44451
I20260811 17:19:42.989555 21946 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:42.989696 21946 heartbeater.cc:511] Master 127.9.157.188:44451 requested a full tablet report, sending...
I20260811 17:19:42.990579 20960 ts_manager.cc:194] Registered new tserver with Master: 4cd592e9c42a4fe8881afa489954dd8e (127.9.157.131:37867)
I20260811 17:19:42.990801 20058 ts_manager.cc:194] Re-registered known tserver with Master: 4cd592e9c42a4fe8881afa489954dd8e (127.9.157.131:37867)
I20260811 17:19:42.990921 21255 ts_manager.cc:194] Registered new tserver with Master: 4cd592e9c42a4fe8881afa489954dd8e (127.9.157.131:37867)
I20260811 17:19:42.991622 20058 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.157.131@KRBTEST.COM'} at 127.9.157.131:40865
I20260811 17:19:42.992362 21963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:42.972069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53076 (local address 127.9.157.190:34565)
0811 17:19:42.973820 (+ 1751us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:42.973837 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:42.973855 (+ 18us) server_negotiation.cc:407] Connection header received
0811 17:19:42.973904 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:42.973909 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:42.973968 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:42.974072 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:42.975272 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.976194 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.977586 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:42.978024 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:42.978128 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:42.987454 (+ 9326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:42.987459 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:42.987463 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:42.987493 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:42.989676 (+ 2183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.990373 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.990377 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.990379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.990486 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:42.990859 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:42.990864 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:42.990866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:42.991109 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:42.991252 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:42.992028 (+ 776us) server_negotiation.cc:299] Negotiation successful
0811 17:19:42.992179 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1642,"thread_start_us":66,"threads_started":1}
I20260811 17:19:42.995626 20058 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53076:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:19:42.999132 20058 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:19:43.002192 20898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "41918f3984d74e9a9e03e35f07467cb1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: NON_VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: true } } }
I20260811 17:19:43.002403 20898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:43.004462 21967 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1: attempting to promote NON_VOTER bac5ec8af44844e388ff31729b5549bc to VOTER
I20260811 17:19:43.004892 21967 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: false } }
I20260811 17:19:43.006104 20972 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [term 1 FOLLOWER]: Refusing update from remote peer 41918f3984d74e9a9e03e35f07467cb1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 17:19:43.006206 21269 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 1 LEARNER]: Refusing update from remote peer 41918f3984d74e9a9e03e35f07467cb1: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 17:19:43.006546 20898 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 17:19:43.006681 21967 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 17:19:43.008597 20898 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, bac5ec8af44844e388ff31729b5549bc (127.9.157.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: false } } }
I20260811 17:19:43.009028 21269 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, bac5ec8af44844e388ff31729b5549bc (127.9.157.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: false } } }
I20260811 17:19:43.010679 21969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "41918f3984d74e9a9e03e35f07467cb1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: false } } }
I20260811 17:19:43.010824 21969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:43.010986 21969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "41918f3984d74e9a9e03e35f07467cb1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: false } } }
I20260811 17:19:43.011075 21969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:43.011013 21315 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "41918f3984d74e9a9e03e35f07467cb1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: false } } }
I20260811 17:19:43.011147 21315 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:19:43.011178 21969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "41918f3984d74e9a9e03e35f07467cb1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: false } } }
I20260811 17:19:43.011245 21969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:43.011593 20972 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, bac5ec8af44844e388ff31729b5549bc (127.9.157.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: false } } }
I20260811 17:19:43.013064 21984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "41918f3984d74e9a9e03e35f07467cb1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: false } } }
I20260811 17:19:43.013204 21984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:19:43.023967 21759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:43.018008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:37833 (local address 127.0.0.1:54422)
0811 17:19:43.018199 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:43.018216 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:43.018306 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:43.019416 (+ 1110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:43.019422 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:43.019446 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:43.019561 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:43.019567 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.021387 (+ 1820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.021392 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:43.022499 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:43.022510 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.023517 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.023521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:43.023607 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:43.023628 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 17:19:43.023729 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 17:19:43.023820 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116}
I20260811 17:19:43.024034 21988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:43.018233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54422 (local address 127.9.157.130:37833)
0811 17:19:43.018639 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:43.018657 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:43.018705 (+ 48us) server_negotiation.cc:407] Connection header received
0811 17:19:43.018800 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:43.018806 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:43.019104 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:43.019827 (+ 723us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:43.020023 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.021274 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.022637 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.023379 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.023517 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:43.023719 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:43.023780 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 17:19:43.023888 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":167,"threads_started":1}
I20260811 17:19:43.024574 21991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:43.018199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:37867 (local address 127.0.0.1:41408)
0811 17:19:43.018838 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:43.018852 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:43.018951 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:43.020225 (+ 1274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:43.020228 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:43.020248 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:43.020351 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:43.020360 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.021901 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.021907 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:43.023072 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:43.023081 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.024188 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.024192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:43.024296 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:43.024325 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 17:19:43.024368 (+ 43us) client_negotiation.cc:240] Negotiation successful
0811 17:19:43.024431 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"mutex_wait_us":30,"thread_start_us":137,"threads_started":1}
I20260811 17:19:43.025316 21994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:43.018704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41408 (local address 127.9.157.131:37867)
0811 17:19:43.019708 (+ 1004us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:43.019720 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:43.019745 (+ 25us) server_negotiation.cc:407] Connection header received
0811 17:19:43.019833 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:43.019838 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:43.020071 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:43.020247 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:43.020457 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.021738 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.023192 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.024049 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.024856 (+ 807us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:43.025001 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:43.025051 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 17:19:43.025143 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":860,"thread_start_us":172,"threads_started":1}
I20260811 17:19:43.027737 21641 tablet_service.cc:1467] Processing CreateTablet for tablet 1f05e6c2583645bea16eacb163544030 (DEFAULT_TABLE table=test-table [id=9c20ad9ebf0d46c0a3bded10ea94a8e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:43.027966 21850 tablet_service.cc:1467] Processing CreateTablet for tablet 1f05e6c2583645bea16eacb163544030 (DEFAULT_TABLE table=test-table [id=9c20ad9ebf0d46c0a3bded10ea94a8e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:43.028841 21641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f05e6c2583645bea16eacb163544030. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:43.028908 21850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f05e6c2583645bea16eacb163544030. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:43.035252 21986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:43.018547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:38203 (local address 127.0.0.1:52652)
0811 17:19:43.019400 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:43.019413 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:43.019519 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:43.020421 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:43.020424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:43.020441 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:43.020527 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:43.020532 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.021934 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.021938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:43.024216 (+ 2278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:43.024224 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.025221 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.025225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:43.025312 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:43.025333 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 17:19:43.034895 (+ 9562us) client_negotiation.cc:240] Negotiation successful
0811 17:19:43.035044 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"thread_start_us":122,"threads_started":1}
I20260811 17:19:43.036172 21995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:43.018992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52652 (local address 127.9.157.129:38203)
0811 17:19:43.019397 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:43.019404 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:43.019781 (+ 377us) server_negotiation.cc:407] Connection header received
0811 17:19:43.019871 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:43.019876 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:43.020116 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:43.020261 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:43.020661 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.021800 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.024328 (+ 2528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.025088 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.025219 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:43.025364 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:43.035740 (+ 10376us) server_negotiation.cc:299] Negotiation successful
0811 17:19:43.036002 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":79,"threads_started":1}
I20260811 17:19:43.037158 21996 tablet_bootstrap.cc:492] T 1f05e6c2583645bea16eacb163544030 P 7432ee13358d444494d13841c91bfd5f: Bootstrap starting.
I20260811 17:19:43.037509 21997 tablet_bootstrap.cc:492] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e: Bootstrap starting.
I20260811 17:19:43.039749 21996 tablet_bootstrap.cc:654] T 1f05e6c2583645bea16eacb163544030 P 7432ee13358d444494d13841c91bfd5f: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:43.039752 21997 tablet_bootstrap.cc:654] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:43.039474 21424 tablet_service.cc:1467] Processing CreateTablet for tablet 1f05e6c2583645bea16eacb163544030 (DEFAULT_TABLE table=test-table [id=9c20ad9ebf0d46c0a3bded10ea94a8e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:43.040637 21996 log.cc:824] T 1f05e6c2583645bea16eacb163544030 P 7432ee13358d444494d13841c91bfd5f: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:43.040618 21997 log.cc:824] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:43.040743 21424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f05e6c2583645bea16eacb163544030. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:43.043179 21997 tablet_bootstrap.cc:492] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e: No bootstrap required, opened a new log
I20260811 17:19:43.043280 21996 tablet_bootstrap.cc:492] T 1f05e6c2583645bea16eacb163544030 P 7432ee13358d444494d13841c91bfd5f: No bootstrap required, opened a new log
I20260811 17:19:43.043444 21997 ts_tablet_manager.cc:1397] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 17:19:43.043474 21996 ts_tablet_manager.cc:1397] T 1f05e6c2583645bea16eacb163544030 P 7432ee13358d444494d13841c91bfd5f: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260811 17:19:43.047363 21997 raft_consensus.cc:348] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7432ee13358d444494d13841c91bfd5f" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37833 } } peers { permanent_uuid: "4cd592e9c42a4fe8881afa489954dd8e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 37867 } } peers { permanent_uuid: "6756578bcb7b438b8a5bcbaaecc2914a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38203 } }
I20260811 17:19:43.047719 21997 raft_consensus.cc:374] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:43.047788 21997 raft_consensus.cc:729] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cd592e9c42a4fe8881afa489954dd8e, State: Initialized, Role: FOLLOWER
I20260811 17:19:43.047680 21996 raft_consensus.cc:348] T 1f05e6c2583645bea16eacb163544030 P 7432ee13358d444494d13841c91bfd5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7432ee13358d444494d13841c91bfd5f" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37833 } } peers { permanent_uuid: "4cd592e9c42a4fe8881afa489954dd8e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 37867 } } peers { permanent_uuid: "6756578bcb7b438b8a5bcbaaecc2914a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38203 } }
I20260811 17:19:43.047951 21996 raft_consensus.cc:374] T 1f05e6c2583645bea16eacb163544030 P 7432ee13358d444494d13841c91bfd5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:43.048023 21996 raft_consensus.cc:729] T 1f05e6c2583645bea16eacb163544030 P 7432ee13358d444494d13841c91bfd5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7432ee13358d444494d13841c91bfd5f, State: Initialized, Role: FOLLOWER
I20260811 17:19:43.048354 21997 consensus_queue.cc:260] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7432ee13358d444494d13841c91bfd5f" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37833 } } peers { permanent_uuid: "4cd592e9c42a4fe8881afa489954dd8e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 37867 } } peers { permanent_uuid: "6756578bcb7b438b8a5bcbaaecc2914a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38203 } }
I20260811 17:19:43.048467 21996 consensus_queue.cc:260] T 1f05e6c2583645bea16eacb163544030 P 7432ee13358d444494d13841c91bfd5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7432ee13358d444494d13841c91bfd5f" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37833 } } peers { permanent_uuid: "4cd592e9c42a4fe8881afa489954dd8e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 37867 } } peers { permanent_uuid: "6756578bcb7b438b8a5bcbaaecc2914a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38203 } }
I20260811 17:19:43.049518 21996 ts_tablet_manager.cc:1428] T 1f05e6c2583645bea16eacb163544030 P 7432ee13358d444494d13841c91bfd5f: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260811 17:19:43.049542 21997 ts_tablet_manager.cc:1428] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260811 17:19:43.049912 21752 heartbeater.cc:503] Master 127.9.157.190:34565 was elected leader, sending a full tablet report...
I20260811 17:19:43.050099 21947 heartbeater.cc:503] Master 127.9.157.190:34565 was elected leader, sending a full tablet report...
I20260811 17:19:43.051537 22004 tablet_bootstrap.cc:492] T 1f05e6c2583645bea16eacb163544030 P 6756578bcb7b438b8a5bcbaaecc2914a: Bootstrap starting.
I20260811 17:19:43.054183 22004 tablet_bootstrap.cc:654] T 1f05e6c2583645bea16eacb163544030 P 6756578bcb7b438b8a5bcbaaecc2914a: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:43.055254 22004 log.cc:824] T 1f05e6c2583645bea16eacb163544030 P 6756578bcb7b438b8a5bcbaaecc2914a: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:43.058640 22004 tablet_bootstrap.cc:492] T 1f05e6c2583645bea16eacb163544030 P 6756578bcb7b438b8a5bcbaaecc2914a: No bootstrap required, opened a new log
I20260811 17:19:43.058830 22004 ts_tablet_manager.cc:1397] T 1f05e6c2583645bea16eacb163544030 P 6756578bcb7b438b8a5bcbaaecc2914a: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 17:19:43.061957 22004 raft_consensus.cc:348] T 1f05e6c2583645bea16eacb163544030 P 6756578bcb7b438b8a5bcbaaecc2914a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7432ee13358d444494d13841c91bfd5f" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37833 } } peers { permanent_uuid: "4cd592e9c42a4fe8881afa489954dd8e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 37867 } } peers { permanent_uuid: "6756578bcb7b438b8a5bcbaaecc2914a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38203 } }
I20260811 17:19:43.062202 22004 raft_consensus.cc:374] T 1f05e6c2583645bea16eacb163544030 P 6756578bcb7b438b8a5bcbaaecc2914a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:43.062237 22004 raft_consensus.cc:729] T 1f05e6c2583645bea16eacb163544030 P 6756578bcb7b438b8a5bcbaaecc2914a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6756578bcb7b438b8a5bcbaaecc2914a, State: Initialized, Role: FOLLOWER
I20260811 17:19:43.062636 22004 consensus_queue.cc:260] T 1f05e6c2583645bea16eacb163544030 P 6756578bcb7b438b8a5bcbaaecc2914a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7432ee13358d444494d13841c91bfd5f" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37833 } } peers { permanent_uuid: "4cd592e9c42a4fe8881afa489954dd8e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 37867 } } peers { permanent_uuid: "6756578bcb7b438b8a5bcbaaecc2914a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38203 } }
I20260811 17:19:43.063380 21531 heartbeater.cc:503] Master 127.9.157.190:34565 was elected leader, sending a full tablet report...
I20260811 17:19:43.063697 22004 ts_tablet_manager.cc:1428] T 1f05e6c2583645bea16eacb163544030 P 6756578bcb7b438b8a5bcbaaecc2914a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 17:19:43.129724 22001 raft_consensus.cc:482] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:19:43.130033 22001 raft_consensus.cc:504] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7432ee13358d444494d13841c91bfd5f" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37833 } } peers { permanent_uuid: "4cd592e9c42a4fe8881afa489954dd8e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 37867 } } peers { permanent_uuid: "6756578bcb7b438b8a5bcbaaecc2914a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38203 } }
I20260811 17:19:43.131361 22001 leader_election.cc:288] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7432ee13358d444494d13841c91bfd5f (127.9.157.130:37833), 6756578bcb7b438b8a5bcbaaecc2914a (127.9.157.129:38203)
I20260811 17:19:43.135175 21954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:43.131727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:37833 (local address 127.9.157.131:41125)
0811 17:19:43.131884 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:43.131904 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:43.132012 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:43.132286 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:43.132289 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:43.132309 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:43.132451 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:43.132456 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.133223 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.133228 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:43.134040 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:43.134049 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.134712 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.134716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:43.134828 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:43.134861 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 17:19:43.134948 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 17:19:43.135027 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"spinlock_wait_cycles":4864}
I20260811 17:19:43.135174 21988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:43.131842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:41125 (local address 127.9.157.130:37833)
0811 17:19:43.132019 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:43.132027 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:43.132049 (+ 22us) server_negotiation.cc:407] Connection header received
0811 17:19:43.132119 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:43.132122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:43.132176 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:43.132252 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:43.132576 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.133100 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.134176 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.134581 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.134718 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:43.134780 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:43.134969 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 17:19:43.135046 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 17:19:43.135432 21951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:43.132122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:38203 (local address 127.9.157.131:33613)
0811 17:19:43.132257 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:43.132271 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:43.132351 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:43.132646 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:43.132649 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:43.132666 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:43.132796 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:43.132809 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.133556 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.133560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:43.134310 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:43.134320 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.134977 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.134981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:43.135098 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:43.135122 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 17:19:43.135215 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 17:19:43.135288 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 17:19:43.135434 21995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:43.132257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:33613 (local address 127.9.157.129:38203)
0811 17:19:43.132432 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:43.132437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:43.132450 (+ 13us) server_negotiation.cc:407] Connection header received
0811 17:19:43.132503 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:43.132505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:43.132546 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:43.132630 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:43.132929 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.133435 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.134450 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.134851 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.134985 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:43.135037 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:43.135236 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 17:19:43.135323 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 17:19:43.136030 21676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f05e6c2583645bea16eacb163544030" candidate_uuid: "4cd592e9c42a4fe8881afa489954dd8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7432ee13358d444494d13841c91bfd5f" is_pre_election: true
I20260811 17:19:43.136072 21459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f05e6c2583645bea16eacb163544030" candidate_uuid: "4cd592e9c42a4fe8881afa489954dd8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6756578bcb7b438b8a5bcbaaecc2914a" is_pre_election: true
I20260811 17:19:43.136409 21459 raft_consensus.cc:2454] T 1f05e6c2583645bea16eacb163544030 P 6756578bcb7b438b8a5bcbaaecc2914a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cd592e9c42a4fe8881afa489954dd8e in term 0.
I20260811 17:19:43.136409 21676 raft_consensus.cc:2454] T 1f05e6c2583645bea16eacb163544030 P 7432ee13358d444494d13841c91bfd5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cd592e9c42a4fe8881afa489954dd8e in term 0.
I20260811 17:19:43.136989 21782 leader_election.cc:302] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cd592e9c42a4fe8881afa489954dd8e, 6756578bcb7b438b8a5bcbaaecc2914a; no voters:
I20260811 17:19:43.137338 22001 raft_consensus.cc:2781] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:19:43.137423 22001 raft_consensus.cc:482] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:19:43.137454 22001 raft_consensus.cc:3037] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:43.138630 22001 raft_consensus.cc:504] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7432ee13358d444494d13841c91bfd5f" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37833 } } peers { permanent_uuid: "4cd592e9c42a4fe8881afa489954dd8e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 37867 } } peers { permanent_uuid: "6756578bcb7b438b8a5bcbaaecc2914a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38203 } }
I20260811 17:19:43.139016 22001 leader_election.cc:288] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e [CANDIDATE]: Term 1 election: Requested vote from peers 7432ee13358d444494d13841c91bfd5f (127.9.157.130:37833), 6756578bcb7b438b8a5bcbaaecc2914a (127.9.157.129:38203)
I20260811 17:19:43.139492 21676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f05e6c2583645bea16eacb163544030" candidate_uuid: "4cd592e9c42a4fe8881afa489954dd8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7432ee13358d444494d13841c91bfd5f"
I20260811 17:19:43.139560 21459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f05e6c2583645bea16eacb163544030" candidate_uuid: "4cd592e9c42a4fe8881afa489954dd8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6756578bcb7b438b8a5bcbaaecc2914a"
I20260811 17:19:43.139616 21676 raft_consensus.cc:3037] T 1f05e6c2583645bea16eacb163544030 P 7432ee13358d444494d13841c91bfd5f [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:43.139765 21459 raft_consensus.cc:3037] T 1f05e6c2583645bea16eacb163544030 P 6756578bcb7b438b8a5bcbaaecc2914a [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:43.141148 21459 raft_consensus.cc:2454] T 1f05e6c2583645bea16eacb163544030 P 6756578bcb7b438b8a5bcbaaecc2914a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd592e9c42a4fe8881afa489954dd8e in term 1.
I20260811 17:19:43.141469 21782 leader_election.cc:302] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cd592e9c42a4fe8881afa489954dd8e, 6756578bcb7b438b8a5bcbaaecc2914a; no voters:
I20260811 17:19:43.141681 21676 raft_consensus.cc:2454] T 1f05e6c2583645bea16eacb163544030 P 7432ee13358d444494d13841c91bfd5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd592e9c42a4fe8881afa489954dd8e in term 1.
I20260811 17:19:43.141731 22001 raft_consensus.cc:2781] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:43.142016 22001 raft_consensus.cc:686] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e [term 1 LEADER]: Becoming Leader. State: Replica: 4cd592e9c42a4fe8881afa489954dd8e, State: Running, Role: LEADER
I20260811 17:19:43.142652 22001 consensus_queue.cc:237] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7432ee13358d444494d13841c91bfd5f" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37833 } } peers { permanent_uuid: "4cd592e9c42a4fe8881afa489954dd8e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 37867 } } peers { permanent_uuid: "6756578bcb7b438b8a5bcbaaecc2914a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38203 } }
I20260811 17:19:43.146420 20058 catalog_manager.cc:5495] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cd592e9c42a4fe8881afa489954dd8e (127.9.157.131). New cstate: current_term: 1 leader_uuid: "4cd592e9c42a4fe8881afa489954dd8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7432ee13358d444494d13841c91bfd5f" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cd592e9c42a4fe8881afa489954dd8e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 37867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6756578bcb7b438b8a5bcbaaecc2914a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38203 } health_report { overall_health: UNKNOWN } } }
I20260811 17:19:43.180397 21994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:43.176514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41410 (local address 127.9.157.131:37867)
0811 17:19:43.176698 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:43.176704 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:43.176722 (+ 18us) server_negotiation.cc:407] Connection header received
0811 17:19:43.176818 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:43.176820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:43.176882 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:43.176995 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:19:43.177363 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.177902 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.178802 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.179047 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.179144 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:43.179465 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:19:43.180017 (+ 552us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:19:43.180127 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:43.180206 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 17:19:43.180267 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 17:19:43.187127 21459 raft_consensus.cc:1261] T 1f05e6c2583645bea16eacb163544030 P 6756578bcb7b438b8a5bcbaaecc2914a [term 1 FOLLOWER]: Refusing update from remote peer 4cd592e9c42a4fe8881afa489954dd8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:43.187162 21676 raft_consensus.cc:1261] T 1f05e6c2583645bea16eacb163544030 P 7432ee13358d444494d13841c91bfd5f [term 1 FOLLOWER]: Refusing update from remote peer 4cd592e9c42a4fe8881afa489954dd8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:43.188221 22009 consensus_queue.cc:1035] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6756578bcb7b438b8a5bcbaaecc2914a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:43.188541 22001 consensus_queue.cc:1035] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7432ee13358d444494d13841c91bfd5f" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 17:19:43.191897 21533 tablet.cc:2369] T 1f05e6c2583645bea16eacb163544030 P 6756578bcb7b438b8a5bcbaaecc2914a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:19:43.202030 22014 mvcc.cc:205] Tried to move back new op lower bound from 7317376135926374400 to 7317376135754940416. Current Snapshot: MvccSnapshot[applied={T|T < 7317376135926374400}]
I20260811 17:19:43.202932 22013 mvcc.cc:205] Tried to move back new op lower bound from 7317376135926374400 to 7317376135754940416. Current Snapshot: MvccSnapshot[applied={T|T < 7317376135926374400}]
I20260811 17:19:43.216135 20057 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:19:43.216325 20057 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:19:43.222455 20057 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 41918f3984d74e9a9e03e35f07467cb1: Sending DeleteTablet for 3 replicas of tablet 1f05e6c2583645bea16eacb163544030
I20260811 17:19:43.223376 21641 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f05e6c2583645bea16eacb163544030 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:43 UTC) from {username='oryx', principal='oryx/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:54422
I20260811 17:19:43.223536 21850 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f05e6c2583645bea16eacb163544030 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:43 UTC) from {username='oryx', principal='oryx/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:41408
I20260811 17:19:43.223585 21424 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f05e6c2583645bea16eacb163544030 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:43 UTC) from {username='oryx', principal='oryx/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:52652
I20260811 17:19:43.224146 22027 tablet_replica.cc:330] T 1f05e6c2583645bea16eacb163544030 P 7432ee13358d444494d13841c91bfd5f: stopping tablet replica
I20260811 17:19:43.224433 22028 tablet_replica.cc:330] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e: stopping tablet replica
I20260811 17:19:43.224570 22027 raft_consensus.cc:2229] T 1f05e6c2583645bea16eacb163544030 P 7432ee13358d444494d13841c91bfd5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:19:43.224785 22029 tablet_replica.cc:330] T 1f05e6c2583645bea16eacb163544030 P 6756578bcb7b438b8a5bcbaaecc2914a: stopping tablet replica
I20260811 17:19:43.224992 22027 raft_consensus.cc:2258] T 1f05e6c2583645bea16eacb163544030 P 7432ee13358d444494d13841c91bfd5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:43.225406 22029 raft_consensus.cc:2229] T 1f05e6c2583645bea16eacb163544030 P 6756578bcb7b438b8a5bcbaaecc2914a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:19:43.225633 22028 raft_consensus.cc:2229] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:19:43.226017 22029 raft_consensus.cc:2258] T 1f05e6c2583645bea16eacb163544030 P 6756578bcb7b438b8a5bcbaaecc2914a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:43.226518 22028 raft_consensus.cc:2258] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:43.226912 22027 ts_tablet_manager.cc:1905] T 1f05e6c2583645bea16eacb163544030 P 7432ee13358d444494d13841c91bfd5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:43.227509 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 20003
I20260811 17:19:43.238165 22029 ts_tablet_manager.cc:1905] T 1f05e6c2583645bea16eacb163544030 P 6756578bcb7b438b8a5bcbaaecc2914a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:43.240254 22028 ts_tablet_manager.cc:1905] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:43.242420 22027 ts_tablet_manager.cc:1918] T 1f05e6c2583645bea16eacb163544030 P 7432ee13358d444494d13841c91bfd5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:19:43.242559 22027 log.cc:1196] T 1f05e6c2583645bea16eacb163544030 P 7432ee13358d444494d13841c91bfd5f: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/wal/wals/1f05e6c2583645bea16eacb163544030
I20260811 17:19:43.242935 22027 ts_tablet_manager.cc:1939] T 1f05e6c2583645bea16eacb163544030 P 7432ee13358d444494d13841c91bfd5f: Deleting consensus metadata
I20260811 17:19:43.245514 22029 ts_tablet_manager.cc:1918] T 1f05e6c2583645bea16eacb163544030 P 6756578bcb7b438b8a5bcbaaecc2914a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:19:43.245662 22029 log.cc:1196] T 1f05e6c2583645bea16eacb163544030 P 6756578bcb7b438b8a5bcbaaecc2914a: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-0/wal/wals/1f05e6c2583645bea16eacb163544030
I20260811 17:19:43.245975 22029 ts_tablet_manager.cc:1939] T 1f05e6c2583645bea16eacb163544030 P 6756578bcb7b438b8a5bcbaaecc2914a: Deleting consensus metadata
I20260811 17:19:43.248034 22028 ts_tablet_manager.cc:1918] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:19:43.248173 22028 log.cc:1196] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/wal/wals/1f05e6c2583645bea16eacb163544030
I20260811 17:19:43.248538 22028 ts_tablet_manager.cc:1939] T 1f05e6c2583645bea16eacb163544030 P 4cd592e9c42a4fe8881afa489954dd8e: Deleting consensus metadata
W20260811 17:19:43.248922 21360 connection.cc:535] server connection from 127.0.0.1:52652 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 17:19:43.249332 21575 connection.cc:535] server connection from 127.0.0.1:54422 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 17:19:43.249951 21782 connection.cc:441] server connection from 127.0.0.1:41408 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41408 (request call id 1) could send its response
I20260811 17:19:43.260878 21960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:43.227200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48696 (local address 127.9.157.189:33385)
0811 17:19:43.227506 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:43.227512 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:43.250082 (+ 22570us) server_negotiation.cc:407] Connection header received
0811 17:19:43.250271 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:43.250277 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:43.250354 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:43.250421 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:43.251513 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.252405 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.253557 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.253906 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.254019 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:43.256131 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:43.256135 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:43.256137 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:43.256167 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:43.258034 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:43.258837 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:43.258841 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:43.258843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:43.258943 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:43.259256 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:43.259262 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:43.259264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:43.259869 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:43.260017 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:43.260522 (+ 505us) server_negotiation.cc:299] Negotiation successful
0811 17:19:43.260670 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206}
I20260811 17:19:43.264902 21760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:43.253321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33748 (local address 127.9.157.188:44451)
0811 17:19:43.253541 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:43.253546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:43.254103 (+ 557us) server_negotiation.cc:407] Connection header received
0811 17:19:43.254289 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:43.254293 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:43.254347 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:43.254408 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:43.255557 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.256375 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.258350 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.258742 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.258826 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:43.260684 (+ 1858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:43.260688 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:43.260690 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:43.260720 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:43.262919 (+ 2199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:43.263441 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:43.263445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:43.263447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:43.263533 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:43.263896 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:43.263900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:43.263901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:43.264116 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:43.264221 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:43.264574 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 17:19:43.264703 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260811 17:19:43.265002 21146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:43.253293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48710 (local address 127.9.157.189:33385)
0811 17:19:43.253412 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:43.253417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:43.253856 (+ 439us) server_negotiation.cc:407] Connection header received
0811 17:19:43.254054 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:43.254058 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:43.254107 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:43.254157 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:43.255429 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.256215 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.257439 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.257783 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.257858 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:43.259628 (+ 1770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:43.259633 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:43.259707 (+ 74us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:43.259734 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:43.263043 (+ 3309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:43.263570 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:43.263573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:43.263575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:43.263686 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:43.264040 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:43.264044 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:43.264046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:43.264275 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:43.264377 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:43.264720 (+ 343us) server_negotiation.cc:299] Negotiation successful
0811 17:19:43.264845 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 17:19:43.270779 21534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:43.225820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33744 (local address 127.9.157.188:44451)
0811 17:19:43.225954 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:43.225959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:43.261323 (+ 35364us) server_negotiation.cc:407] Connection header received
0811 17:19:43.261526 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:43.261530 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:43.261580 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:43.261633 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:43.262469 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.263259 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.264403 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:43.264719 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:43.264793 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:43.266697 (+ 1904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:43.266701 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:43.266703 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:43.266731 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:43.268616 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:43.269467 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:43.269471 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:43.269473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:43.269564 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:43.269855 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:43.269862 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:43.269865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:43.270072 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:43.270217 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:43.270474 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 17:19:43.270604 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 17:19:44.204130 22042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:44.203554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.9.157.129:42057)
0811 17:19:44.203875 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:44.203994 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":107,"threads_started":1}
W20260811 17:19:44.204650 21531 heartbeater.cc:644] Failed to heartbeat to 127.9.157.190:34565 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
I20260811 17:19:44.205114 22043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:44.204618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.9.157.131:57249)
0811 17:19:44.204895 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:44.205009 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":77,"threads_started":1}
W20260811 17:19:44.205680 21947 heartbeater.cc:644] Failed to heartbeat to 127.9.157.190:34565 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
I20260811 17:19:44.205837 22044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:44.205377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.9.157.130:55607)
0811 17:19:44.205634 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:44.205724 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":76,"threads_started":1}
W20260811 17:19:44.206110 21752 heartbeater.cc:644] Failed to heartbeat to 127.9.157.190:34565 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
I20260811 17:19:44.722070 22046 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41918f3984d74e9a9e03e35f07467cb1)
I20260811 17:19:44.722329 22049 raft_consensus.cc:482] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41918f3984d74e9a9e03e35f07467cb1)
I20260811 17:19:44.722393 22046 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: false } }
I20260811 17:19:44.722566 22049 raft_consensus.cc:504] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: false } }
I20260811 17:19:44.723611 22049 leader_election.cc:288] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41918f3984d74e9a9e03e35f07467cb1 (127.9.157.190:34565), 2409126f018b4ea38b67722ebcfdc44d (127.9.157.189:33385)
I20260811 17:19:44.723847 22046 leader_election.cc:288] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41918f3984d74e9a9e03e35f07467cb1 (127.9.157.190:34565), bac5ec8af44844e388ff31729b5549bc (127.9.157.188:44451)
I20260811 17:19:44.724403 22052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:44.723904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.0.0.1:53550)
0811 17:19:44.724182 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:44.724276 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":77,"threads_started":1}
W20260811 17:19:44.724761 20934 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
I20260811 17:19:44.725804 22056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:44.725395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.0.0.1:53540)
0811 17:19:44.725630 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:44.725708 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":84,"threads_started":1}
I20260811 17:19:44.726001 22054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:44.725458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.0.0.1:53558)
0811 17:19:44.725856 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:44.725918 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336}
W20260811 17:19:44.726030 21237 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
W20260811 17:19:44.726135 20934 leader_election.cc:334] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41918f3984d74e9a9e03e35f07467cb1 (127.9.157.190:34565): Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
I20260811 17:19:44.727110 22056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:44.726797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.0.0.1:53570)
0811 17:19:44.726980 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:44.727035 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260811 17:19:44.727249 21237 leader_election.cc:334] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41918f3984d74e9a9e03e35f07467cb1 (127.9.157.190:34565): Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
I20260811 17:19:44.729528 22053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:44.724137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48720 (local address 127.9.157.189:33385)
0811 17:19:44.724354 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:44.724358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:44.724375 (+ 17us) server_negotiation.cc:407] Connection header received
0811 17:19:44.724426 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:44.724430 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:44.724602 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:44.724691 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:44.725190 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:44.726205 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:44.728104 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:44.728705 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:44.728831 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:44.728905 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:44.729289 (+ 384us) server_negotiation.cc:299] Negotiation successful
0811 17:19:44.729379 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":72,"threads_started":1}
I20260811 17:19:44.729975 22050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:44.723894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:33385 (local address 127.0.0.1:48720)
0811 17:19:44.724138 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:44.724156 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:44.724285 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:44.724848 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:44.724851 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:44.724923 (+ 72us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:44.725045 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:44.725052 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:44.726403 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:44.726407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:44.727953 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:44.727964 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:44.728974 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:44.728979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:44.729084 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:44.729123 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 17:19:44.729194 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 17:19:44.729823 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":549,"thread_start_us":77,"threads_started":1}
I20260811 17:19:44.730355 22052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:44.724408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:44451 (local address 127.0.0.1:33750)
0811 17:19:44.724548 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:44.724561 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:44.724648 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:44.726421 (+ 1773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:44.726424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:44.726469 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:44.726558 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:44.726563 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:44.727748 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:44.727752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:44.729012 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:44.729021 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:44.729919 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:44.729923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:44.730031 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:44.730074 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 17:19:44.730130 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 17:19:44.730208 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":140,"threads_started":1}
I20260811 17:19:44.730840 22057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:44.725641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33750 (local address 127.9.157.188:44451)
0811 17:19:44.725858 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:44.725863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:44.725879 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:19:44.725926 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:44.725929 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:44.725971 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:44.726061 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:44.726666 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:44.727574 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:44.729149 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:44.729765 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:44.730546 (+ 781us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:44.730610 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:44.730652 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 17:19:44.730732 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":148,"thread_start_us":68,"threads_started":1}
I20260811 17:19:44.730892 20972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bac5ec8af44844e388ff31729b5549bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2409126f018b4ea38b67722ebcfdc44d" is_pre_election: true
I20260811 17:19:44.731166 20972 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bac5ec8af44844e388ff31729b5549bc in term 1.
I20260811 17:19:44.731312 21269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2409126f018b4ea38b67722ebcfdc44d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bac5ec8af44844e388ff31729b5549bc" is_pre_election: true
I20260811 17:19:44.731534 21269 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2409126f018b4ea38b67722ebcfdc44d in term 1.
I20260811 17:19:44.731617 21236 leader_election.cc:302] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2409126f018b4ea38b67722ebcfdc44d, bac5ec8af44844e388ff31729b5549bc; no voters: 41918f3984d74e9a9e03e35f07467cb1
I20260811 17:19:44.731931 20934 leader_election.cc:302] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2409126f018b4ea38b67722ebcfdc44d, bac5ec8af44844e388ff31729b5549bc; no voters: 41918f3984d74e9a9e03e35f07467cb1
I20260811 17:19:44.732098 22049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 17:19:44.732175 22049 raft_consensus.cc:482] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41918f3984d74e9a9e03e35f07467cb1)
I20260811 17:19:44.732216 22049 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 1 FOLLOWER]: Advancing to term 2
I20260811 17:19:44.732321 22046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 17:19:44.732369 22046 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41918f3984d74e9a9e03e35f07467cb1)
I20260811 17:19:44.732422 22046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [term 1 FOLLOWER]: Advancing to term 2
I20260811 17:19:44.733657 22046 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: false } }
I20260811 17:19:44.733657 22049 raft_consensus.cc:504] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: false } }
I20260811 17:19:44.734097 22046 leader_election.cc:288] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [CANDIDATE]: Term 2 election: Requested vote from peers 41918f3984d74e9a9e03e35f07467cb1 (127.9.157.190:34565), bac5ec8af44844e388ff31729b5549bc (127.9.157.188:44451)
I20260811 17:19:44.734115 22049 leader_election.cc:288] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [CANDIDATE]: Term 2 election: Requested vote from peers 41918f3984d74e9a9e03e35f07467cb1 (127.9.157.190:34565), 2409126f018b4ea38b67722ebcfdc44d (127.9.157.189:33385)
I20260811 17:19:44.734738 20972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bac5ec8af44844e388ff31729b5549bc" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2409126f018b4ea38b67722ebcfdc44d"
I20260811 17:19:44.734788 22052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:44.734406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.0.0.1:53582)
0811 17:19:44.734597 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:44.734677 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 17:19:44.734911 20972 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate bac5ec8af44844e388ff31729b5549bc in current term 2: Already voted for candidate 2409126f018b4ea38b67722ebcfdc44d in this term.
I20260811 17:19:44.735119 21269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2409126f018b4ea38b67722ebcfdc44d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bac5ec8af44844e388ff31729b5549bc"
I20260811 17:19:44.735357 21269 raft_consensus.cc:2379] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 2409126f018b4ea38b67722ebcfdc44d in current term 2: Already voted for candidate bac5ec8af44844e388ff31729b5549bc in this term.
I20260811 17:19:44.735715 22052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:44.735380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.0.0.1:53590)
0811 17:19:44.735532 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:44.735590 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260811 17:19:44.735880 20934 leader_election.cc:334] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 41918f3984d74e9a9e03e35f07467cb1 (127.9.157.190:34565): Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
I20260811 17:19:44.735978 22050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:44.734347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.0.0.1:53572)
0811 17:19:44.735809 (+ 1462us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:44.735882 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1360}
I20260811 17:19:44.736006 20934 leader_election.cc:302] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2409126f018b4ea38b67722ebcfdc44d; no voters: 41918f3984d74e9a9e03e35f07467cb1, bac5ec8af44844e388ff31729b5549bc
I20260811 17:19:44.736300 22046 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 17:19:44.736901 22050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:44.736604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.0.0.1:53596)
0811 17:19:44.736753 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:44.736818 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 17:19:44.737021 21237 leader_election.cc:334] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 41918f3984d74e9a9e03e35f07467cb1 (127.9.157.190:34565): Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
I20260811 17:19:44.737114 21237 leader_election.cc:302] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bac5ec8af44844e388ff31729b5549bc; no voters: 2409126f018b4ea38b67722ebcfdc44d, 41918f3984d74e9a9e03e35f07467cb1
I20260811 17:19:44.737298 22049 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 17:19:45.206331 22060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:45.205612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.9.157.129:47665)
0811 17:19:45.206078 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:45.206167 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":105,"threads_started":1}
I20260811 17:19:45.206956 22061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:45.206301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.9.157.131:39977)
0811 17:19:45.206763 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:45.206838 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":90,"threads_started":1}
I20260811 17:19:45.207123 22062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:45.206658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.9.157.130:53825)
0811 17:19:45.206947 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:45.207017 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":79,"threads_started":1}
I20260811 17:19:46.207845 22063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.207140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.9.157.129:57123)
0811 17:19:46.207619 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:46.207724 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":93,"threads_started":1}
I20260811 17:19:46.208298 22068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.207802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.9.157.131:48673)
0811 17:19:46.208126 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:46.208190 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":73,"threads_started":1}
I20260811 17:19:46.208331 22066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.207802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.9.157.130:47065)
0811 17:19:46.208195 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:46.208242 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":70,"threads_started":1}
I20260811 17:19:46.452139 22069 raft_consensus.cc:482] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:19:46.452280 22069 raft_consensus.cc:504] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: false } }
I20260811 17:19:46.452899 22069 leader_election.cc:288] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 41918f3984d74e9a9e03e35f07467cb1 (127.9.157.190:34565), 2409126f018b4ea38b67722ebcfdc44d (127.9.157.189:33385)
I20260811 17:19:46.453436 20972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bac5ec8af44844e388ff31729b5549bc" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2409126f018b4ea38b67722ebcfdc44d" is_pre_election: true
I20260811 17:19:46.453655 20972 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bac5ec8af44844e388ff31729b5549bc in term 2.
I20260811 17:19:46.453716 22070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.453128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.0.0.1:35028)
0811 17:19:46.453467 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:46.453581 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":69,"threads_started":1}
I20260811 17:19:46.454037 21236 leader_election.cc:302] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2409126f018b4ea38b67722ebcfdc44d, bac5ec8af44844e388ff31729b5549bc; no voters:
I20260811 17:19:46.454463 22069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 17:19:46.454587 22069 raft_consensus.cc:482] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:19:46.454625 22070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.454330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.0.0.1:35030)
0811 17:19:46.454469 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:46.454532 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 17:19:46.454640 22069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 2 FOLLOWER]: Advancing to term 3
W20260811 17:19:46.454710 21237 leader_election.cc:334] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 41918f3984d74e9a9e03e35f07467cb1 (127.9.157.190:34565): Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
I20260811 17:19:46.455904 22069 raft_consensus.cc:504] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: false } }
I20260811 17:19:46.456278 22069 leader_election.cc:288] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [CANDIDATE]: Term 3 election: Requested vote from peers 41918f3984d74e9a9e03e35f07467cb1 (127.9.157.190:34565), 2409126f018b4ea38b67722ebcfdc44d (127.9.157.189:33385)
I20260811 17:19:46.456614 22070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.456410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.0.0.1:35034)
0811 17:19:46.456497 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:46.456562 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260811 17:19:46.456704 20972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bac5ec8af44844e388ff31729b5549bc" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2409126f018b4ea38b67722ebcfdc44d"
I20260811 17:19:46.456889 20972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [term 2 FOLLOWER]: Advancing to term 3
I20260811 17:19:46.457310 22070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.457101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.0.0.1:35038)
0811 17:19:46.457207 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:46.457255 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 17:19:46.457492 21237 leader_election.cc:334] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 41918f3984d74e9a9e03e35f07467cb1 (127.9.157.190:34565): Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
I20260811 17:19:46.458055 20972 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bac5ec8af44844e388ff31729b5549bc in term 3.
I20260811 17:19:46.458351 21236 leader_election.cc:302] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2409126f018b4ea38b67722ebcfdc44d, bac5ec8af44844e388ff31729b5549bc; no voters: 41918f3984d74e9a9e03e35f07467cb1
I20260811 17:19:46.458515 22069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 3 FOLLOWER]: Leader election won for term 3
I20260811 17:19:46.458770 22069 raft_consensus.cc:686] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [term 3 LEADER]: Becoming Leader. State: Replica: bac5ec8af44844e388ff31729b5549bc, State: Running, Role: LEADER
I20260811 17:19:46.459057 22069 consensus_queue.cc:237] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: false } }
I20260811 17:19:46.460239 22073 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader bac5ec8af44844e388ff31729b5549bc. Latest consensus state: current_term: 3 leader_uuid: "bac5ec8af44844e388ff31729b5549bc" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: false } } }
I20260811 17:19:46.460379 22073 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:46.460731 22079 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:19:46.463421 22079 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:19:46.463979 22079 catalog_manager.cc:1259] Loaded cluster ID: 7725b3cbf87546e186272ad03fb69783
I20260811 17:19:46.464031 22079 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:19:46.464679 22079 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:19:46.465131 22079 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc: Loaded TSK: 0
I20260811 17:19:46.465925 22079 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:19:46.507450 21254 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33748:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:19:46.510758 21254 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:19:46.514019 22070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.513710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.0.0.1:35046)
0811 17:19:46.513867 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:46.513934 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 17:19:46.514091 20972 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [term 3 FOLLOWER]: Refusing update from remote peer bac5ec8af44844e388ff31729b5549bc: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260811 17:19:46.514679 22069 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 17:19:46.514861 22070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.514590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.0.0.1:35052)
0811 17:19:46.514709 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:46.514766 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 17:19:46.515048 21237 consensus_peers.cc:487] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc -> Peer 41918f3984d74e9a9e03e35f07467cb1 (127.9.157.190:34565): Couldn't send request to peer 41918f3984d74e9a9e03e35f07467cb1. Status: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 17:19:46.516392 22082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [sys.catalog]: SysCatalogTable state changed. Reason: New leader bac5ec8af44844e388ff31729b5549bc. Latest consensus state: current_term: 3 leader_uuid: "bac5ec8af44844e388ff31729b5549bc" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: false } } }
I20260811 17:19:46.516543 22082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:19:46.517844 22069 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "bac5ec8af44844e388ff31729b5549bc" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: false } } }
I20260811 17:19:46.517974 22069 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:46.518494 22083 mvcc.cc:205] Tried to move back new op lower bound from 7317376149556420608 to 7317376149339115520. Current Snapshot: MvccSnapshot[applied={T|T < 7317376149556420608}]
I20260811 17:19:46.518658 22082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "bac5ec8af44844e388ff31729b5549bc" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "41918f3984d74e9a9e03e35f07467cb1" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34565 } } peers { permanent_uuid: "2409126f018b4ea38b67722ebcfdc44d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 33385 } attrs { promote: false } } peers { permanent_uuid: "bac5ec8af44844e388ff31729b5549bc" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 44451 } attrs { promote: false } } }
I20260811 17:19:46.518777 22082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2409126f018b4ea38b67722ebcfdc44d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:19:46.528411 22070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.524021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:37867 (local address 127.0.0.1:38790)
0811 17:19:46.524183 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:46.524199 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:46.524296 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:46.524713 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:46.524716 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:46.524731 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:46.524854 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:46.524862 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:46.525856 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:46.525862 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:46.527089 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:46.527101 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:46.528006 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:46.528009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:46.528095 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:46.528124 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 17:19:46.528199 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 17:19:46.528274 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 17:19:46.528539 22089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.524129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38790 (local address 127.9.157.131:37867)
0811 17:19:46.524486 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:46.524491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:46.524506 (+ 15us) server_negotiation.cc:407] Connection header received
0811 17:19:46.524568 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:46.524571 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:46.524627 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:46.524787 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:46.524949 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:46.525718 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:46.527224 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:46.527892 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:46.528006 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:46.528075 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:46.528241 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 17:19:46.528348 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":92,"threads_started":1}
I20260811 17:19:46.529076 22090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.524288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:38203 (local address 127.0.0.1:48742)
0811 17:19:46.524490 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:46.524503 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:46.524579 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:46.525704 (+ 1125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:46.525706 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:46.525720 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:46.525821 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:46.525826 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:46.526662 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:46.526668 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:46.527798 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:46.527808 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:46.528697 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:46.528701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:46.528799 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:46.528821 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 17:19:46.528888 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 17:19:46.528950 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":85,"threads_started":1}
I20260811 17:19:46.529050 21850 tablet_service.cc:1467] Processing CreateTablet for tablet befb3c92061743a191350097b75114da (DEFAULT_TABLE table=test-table [id=81157c8688a646c4abae67e43462459c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:46.529357 21850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet befb3c92061743a191350097b75114da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:46.529579 22092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.524627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48742 (local address 127.9.157.129:38203)
0811 17:19:46.525213 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:46.525218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:46.525491 (+ 273us) server_negotiation.cc:407] Connection header received
0811 17:19:46.525557 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:46.525560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:46.525616 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:46.525700 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:46.525929 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:46.526547 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:46.527917 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:46.528551 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:46.528698 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:46.528763 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:46.529342 (+ 579us) server_negotiation.cc:299] Negotiation successful
0811 17:19:46.529450 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":65,"threads_started":1}
I20260811 17:19:46.530217 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.525328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54654 (local address 127.9.157.130:37833)
0811 17:19:46.525580 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:46.525585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:46.525600 (+ 15us) server_negotiation.cc:407] Connection header received
0811 17:19:46.525649 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:46.525652 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:46.525693 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:46.525797 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:46.526177 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:46.527000 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:46.528925 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:46.529530 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:46.529652 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:46.529753 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:46.530003 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 17:19:46.530084 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":83,"threads_started":1}
I20260811 17:19:46.530771 22093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.525194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:37833 (local address 127.0.0.1:54654)
0811 17:19:46.525450 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:46.525465 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:46.525560 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:46.525930 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:46.525932 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:46.525950 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:46.526043 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:46.526049 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:46.527140 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:46.527147 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:46.528337 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:46.528348 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:46.529807 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:46.529810 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:46.529874 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:46.529893 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 17:19:46.529956 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 17:19:46.530058 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":98,"threads_started":1}
I20260811 17:19:46.532089 22095 tablet_bootstrap.cc:492] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e: Bootstrap starting.
I20260811 17:19:46.532455 21424 tablet_service.cc:1467] Processing CreateTablet for tablet befb3c92061743a191350097b75114da (DEFAULT_TABLE table=test-table [id=81157c8688a646c4abae67e43462459c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:46.532748 21424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet befb3c92061743a191350097b75114da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:46.532677 21641 tablet_service.cc:1467] Processing CreateTablet for tablet befb3c92061743a191350097b75114da (DEFAULT_TABLE table=test-table [id=81157c8688a646c4abae67e43462459c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:46.532984 21641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet befb3c92061743a191350097b75114da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:46.533176 22095 tablet_bootstrap.cc:654] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:46.534973 22095 tablet_bootstrap.cc:492] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e: No bootstrap required, opened a new log
I20260811 17:19:46.535071 22095 ts_tablet_manager.cc:1397] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 17:19:46.535344 22099 tablet_bootstrap.cc:492] T befb3c92061743a191350097b75114da P 6756578bcb7b438b8a5bcbaaecc2914a: Bootstrap starting.
I20260811 17:19:46.535561 22095 raft_consensus.cc:348] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cd592e9c42a4fe8881afa489954dd8e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 37867 } } peers { permanent_uuid: "6756578bcb7b438b8a5bcbaaecc2914a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38203 } } peers { permanent_uuid: "7432ee13358d444494d13841c91bfd5f" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37833 } }
I20260811 17:19:46.535732 22095 raft_consensus.cc:374] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:46.535780 22095 raft_consensus.cc:729] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cd592e9c42a4fe8881afa489954dd8e, State: Initialized, Role: FOLLOWER
I20260811 17:19:46.535909 22100 tablet_bootstrap.cc:492] T befb3c92061743a191350097b75114da P 7432ee13358d444494d13841c91bfd5f: Bootstrap starting.
I20260811 17:19:46.535949 22095 consensus_queue.cc:260] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cd592e9c42a4fe8881afa489954dd8e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 37867 } } peers { permanent_uuid: "6756578bcb7b438b8a5bcbaaecc2914a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38203 } } peers { permanent_uuid: "7432ee13358d444494d13841c91bfd5f" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37833 } }
I20260811 17:19:46.536386 22095 ts_tablet_manager.cc:1428] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 17:19:46.536543 22099 tablet_bootstrap.cc:654] T befb3c92061743a191350097b75114da P 6756578bcb7b438b8a5bcbaaecc2914a: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:46.536919 22100 tablet_bootstrap.cc:654] T befb3c92061743a191350097b75114da P 7432ee13358d444494d13841c91bfd5f: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:46.537184 22068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.536791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.9.157.131:33955)
0811 17:19:46.537046 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:46.537109 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173}
I20260811 17:19:46.537909 22068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.537646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.9.157.131:33779)
0811 17:19:46.537788 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:46.537844 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 17:19:46.538410 22099 tablet_bootstrap.cc:492] T befb3c92061743a191350097b75114da P 6756578bcb7b438b8a5bcbaaecc2914a: No bootstrap required, opened a new log
I20260811 17:19:46.538515 22099 ts_tablet_manager.cc:1397] T befb3c92061743a191350097b75114da P 6756578bcb7b438b8a5bcbaaecc2914a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 17:19:46.538942 22099 raft_consensus.cc:348] T befb3c92061743a191350097b75114da P 6756578bcb7b438b8a5bcbaaecc2914a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cd592e9c42a4fe8881afa489954dd8e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 37867 } } peers { permanent_uuid: "6756578bcb7b438b8a5bcbaaecc2914a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38203 } } peers { permanent_uuid: "7432ee13358d444494d13841c91bfd5f" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37833 } }
I20260811 17:19:46.539120 22099 raft_consensus.cc:374] T befb3c92061743a191350097b75114da P 6756578bcb7b438b8a5bcbaaecc2914a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:46.539167 22099 raft_consensus.cc:729] T befb3c92061743a191350097b75114da P 6756578bcb7b438b8a5bcbaaecc2914a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6756578bcb7b438b8a5bcbaaecc2914a, State: Initialized, Role: FOLLOWER
I20260811 17:19:46.539327 22099 consensus_queue.cc:260] T befb3c92061743a191350097b75114da P 6756578bcb7b438b8a5bcbaaecc2914a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cd592e9c42a4fe8881afa489954dd8e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 37867 } } peers { permanent_uuid: "6756578bcb7b438b8a5bcbaaecc2914a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38203 } } peers { permanent_uuid: "7432ee13358d444494d13841c91bfd5f" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37833 } }
I20260811 17:19:46.539471 22100 tablet_bootstrap.cc:492] T befb3c92061743a191350097b75114da P 7432ee13358d444494d13841c91bfd5f: No bootstrap required, opened a new log
I20260811 17:19:46.539567 22100 ts_tablet_manager.cc:1397] T befb3c92061743a191350097b75114da P 7432ee13358d444494d13841c91bfd5f: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 17:19:46.539760 22099 ts_tablet_manager.cc:1428] T befb3c92061743a191350097b75114da P 6756578bcb7b438b8a5bcbaaecc2914a: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 17:19:46.539803 21946 heartbeater.cc:503] Master 127.9.157.188:44451 was elected leader, sending a full tablet report...
I20260811 17:19:46.540046 22100 raft_consensus.cc:348] T befb3c92061743a191350097b75114da P 7432ee13358d444494d13841c91bfd5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cd592e9c42a4fe8881afa489954dd8e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 37867 } } peers { permanent_uuid: "6756578bcb7b438b8a5bcbaaecc2914a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38203 } } peers { permanent_uuid: "7432ee13358d444494d13841c91bfd5f" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37833 } }
I20260811 17:19:46.540167 22100 raft_consensus.cc:374] T befb3c92061743a191350097b75114da P 7432ee13358d444494d13841c91bfd5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:46.540205 22100 raft_consensus.cc:729] T befb3c92061743a191350097b75114da P 7432ee13358d444494d13841c91bfd5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7432ee13358d444494d13841c91bfd5f, State: Initialized, Role: FOLLOWER
I20260811 17:19:46.540338 22100 consensus_queue.cc:260] T befb3c92061743a191350097b75114da P 7432ee13358d444494d13841c91bfd5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cd592e9c42a4fe8881afa489954dd8e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 37867 } } peers { permanent_uuid: "6756578bcb7b438b8a5bcbaaecc2914a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38203 } } peers { permanent_uuid: "7432ee13358d444494d13841c91bfd5f" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37833 } }
I20260811 17:19:46.540776 22100 ts_tablet_manager.cc:1428] T befb3c92061743a191350097b75114da P 7432ee13358d444494d13841c91bfd5f: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260811 17:19:46.541690 22066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.541252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.9.157.130:45163)
0811 17:19:46.541549 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:46.541615 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223}
I20260811 17:19:46.542174 22063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.541899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.9.157.129:35615)
0811 17:19:46.542025 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:46.542094 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 17:19:46.543036 21529 heartbeater.cc:503] Master 127.9.157.188:44451 was elected leader, sending a full tablet report...
I20260811 17:19:46.543143 21747 heartbeater.cc:503] Master 127.9.157.188:44451 was elected leader, sending a full tablet report...
I20260811 17:19:46.543231 22066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.542889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.9.157.130:34705)
0811 17:19:46.543116 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:46.543168 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
I20260811 17:19:46.543941 22063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.543692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.9.157.129:48209)
0811 17:19:46.543823 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:46.543876 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260811 17:19:46.710027 21754 tablet.cc:2369] T befb3c92061743a191350097b75114da P 7432ee13358d444494d13841c91bfd5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:19:46.715817 21949 tablet.cc:2369] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:19:46.778903 22101 raft_consensus.cc:482] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:19:46.779114 22101 raft_consensus.cc:504] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cd592e9c42a4fe8881afa489954dd8e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 37867 } } peers { permanent_uuid: "6756578bcb7b438b8a5bcbaaecc2914a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38203 } } peers { permanent_uuid: "7432ee13358d444494d13841c91bfd5f" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37833 } }
I20260811 17:19:46.779712 22101 leader_election.cc:288] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6756578bcb7b438b8a5bcbaaecc2914a (127.9.157.129:38203), 7432ee13358d444494d13841c91bfd5f (127.9.157.130:37833)
I20260811 17:19:46.780368 21459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "befb3c92061743a191350097b75114da" candidate_uuid: "4cd592e9c42a4fe8881afa489954dd8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6756578bcb7b438b8a5bcbaaecc2914a" is_pre_election: true
I20260811 17:19:46.780367 21676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "befb3c92061743a191350097b75114da" candidate_uuid: "4cd592e9c42a4fe8881afa489954dd8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7432ee13358d444494d13841c91bfd5f" is_pre_election: true
I20260811 17:19:46.780542 21459 raft_consensus.cc:2454] T befb3c92061743a191350097b75114da P 6756578bcb7b438b8a5bcbaaecc2914a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cd592e9c42a4fe8881afa489954dd8e in term 0.
I20260811 17:19:46.780550 21676 raft_consensus.cc:2454] T befb3c92061743a191350097b75114da P 7432ee13358d444494d13841c91bfd5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cd592e9c42a4fe8881afa489954dd8e in term 0.
I20260811 17:19:46.780923 21782 leader_election.cc:302] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cd592e9c42a4fe8881afa489954dd8e, 7432ee13358d444494d13841c91bfd5f; no voters:
I20260811 17:19:46.781123 22101 raft_consensus.cc:2781] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:19:46.781190 22101 raft_consensus.cc:482] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:19:46.781229 22101 raft_consensus.cc:3037] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:46.782359 22101 raft_consensus.cc:504] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cd592e9c42a4fe8881afa489954dd8e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 37867 } } peers { permanent_uuid: "6756578bcb7b438b8a5bcbaaecc2914a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38203 } } peers { permanent_uuid: "7432ee13358d444494d13841c91bfd5f" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37833 } }
I20260811 17:19:46.782728 22101 leader_election.cc:288] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e [CANDIDATE]: Term 1 election: Requested vote from peers 6756578bcb7b438b8a5bcbaaecc2914a (127.9.157.129:38203), 7432ee13358d444494d13841c91bfd5f (127.9.157.130:37833)
I20260811 17:19:46.783183 21676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "befb3c92061743a191350097b75114da" candidate_uuid: "4cd592e9c42a4fe8881afa489954dd8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7432ee13358d444494d13841c91bfd5f"
I20260811 17:19:46.783183 21459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "befb3c92061743a191350097b75114da" candidate_uuid: "4cd592e9c42a4fe8881afa489954dd8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6756578bcb7b438b8a5bcbaaecc2914a"
I20260811 17:19:46.783329 21676 raft_consensus.cc:3037] T befb3c92061743a191350097b75114da P 7432ee13358d444494d13841c91bfd5f [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:46.783329 21459 raft_consensus.cc:3037] T befb3c92061743a191350097b75114da P 6756578bcb7b438b8a5bcbaaecc2914a [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:46.784559 21676 raft_consensus.cc:2454] T befb3c92061743a191350097b75114da P 7432ee13358d444494d13841c91bfd5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd592e9c42a4fe8881afa489954dd8e in term 1.
I20260811 17:19:46.784559 21459 raft_consensus.cc:2454] T befb3c92061743a191350097b75114da P 6756578bcb7b438b8a5bcbaaecc2914a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd592e9c42a4fe8881afa489954dd8e in term 1.
I20260811 17:19:46.784899 21782 leader_election.cc:302] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cd592e9c42a4fe8881afa489954dd8e, 7432ee13358d444494d13841c91bfd5f; no voters:
I20260811 17:19:46.785077 22101 raft_consensus.cc:2781] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:46.785321 22101 raft_consensus.cc:686] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e [term 1 LEADER]: Becoming Leader. State: Replica: 4cd592e9c42a4fe8881afa489954dd8e, State: Running, Role: LEADER
I20260811 17:19:46.785494 22101 consensus_queue.cc:237] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cd592e9c42a4fe8881afa489954dd8e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 37867 } } peers { permanent_uuid: "6756578bcb7b438b8a5bcbaaecc2914a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38203 } } peers { permanent_uuid: "7432ee13358d444494d13841c91bfd5f" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37833 } }
I20260811 17:19:46.786149 22068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.785869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.9.157.131:38115)
0811 17:19:46.786003 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:46.786069 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 17:19:46.787547 21253 catalog_manager.cc:5495] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cd592e9c42a4fe8881afa489954dd8e (127.9.157.131). New cstate: current_term: 1 leader_uuid: "4cd592e9c42a4fe8881afa489954dd8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cd592e9c42a4fe8881afa489954dd8e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 37867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6756578bcb7b438b8a5bcbaaecc2914a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7432ee13358d444494d13841c91bfd5f" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37833 } health_report { overall_health: UNKNOWN } } }
I20260811 17:19:46.824954 22089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.821596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38796 (local address 127.9.157.131:37867)
0811 17:19:46.821750 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:46.821754 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:46.821768 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:19:46.821820 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:46.821822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:46.821869 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:46.821994 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:19:46.822303 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:46.822871 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:46.823811 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:46.824057 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:46.824173 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:46.824443 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:19:46.824538 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:19:46.824618 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:46.824741 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 17:19:46.824806 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 17:19:46.827497 21676 raft_consensus.cc:1261] T befb3c92061743a191350097b75114da P 7432ee13358d444494d13841c91bfd5f [term 1 FOLLOWER]: Refusing update from remote peer 4cd592e9c42a4fe8881afa489954dd8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:46.827647 21459 raft_consensus.cc:1261] T befb3c92061743a191350097b75114da P 6756578bcb7b438b8a5bcbaaecc2914a [term 1 FOLLOWER]: Refusing update from remote peer 4cd592e9c42a4fe8881afa489954dd8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:46.828132 22101 consensus_queue.cc:1035] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7432ee13358d444494d13841c91bfd5f" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:46.828330 22109 consensus_queue.cc:1035] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6756578bcb7b438b8a5bcbaaecc2914a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 38203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:46.830387 22063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.830024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.9.157.129:46991)
0811 17:19:46.830206 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:46.830293 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260811 17:19:46.830761 22066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.830137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.9.157.130:43265)
0811 17:19:46.830604 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:46.830676 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379}
I20260811 17:19:46.833407 22068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.833104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.9.157.131:45761)
0811 17:19:46.833266 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:46.833322 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260811 17:19:46.834566 22063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.834213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.9.157.129:45663)
0811 17:19:46.834433 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:46.834489 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
I20260811 17:19:46.835928 22068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.835619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.9.157.131:58773)
0811 17:19:46.835797 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:46.835856 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260811 17:19:46.838698 22066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.838189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34565 (local address 127.9.157.130:58069)
0811 17:19:46.838407 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:46.838461 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
I20260811 17:19:46.840965 21254 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:19:46.841116 21254 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:19:46.845392 21254 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bac5ec8af44844e388ff31729b5549bc: Sending DeleteTablet for 3 replicas of tablet befb3c92061743a191350097b75114da
I20260811 17:19:46.846143 21850 tablet_service.cc:1514] Processing DeleteTablet for tablet befb3c92061743a191350097b75114da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:46 UTC) from {username='oryx', principal='oryx/127.9.157.188@KRBTEST.COM'} at 127.0.0.1:38790
I20260811 17:19:46.846487 21424 tablet_service.cc:1514] Processing DeleteTablet for tablet befb3c92061743a191350097b75114da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:46 UTC) from {username='oryx', principal='oryx/127.9.157.188@KRBTEST.COM'} at 127.0.0.1:48742
I20260811 17:19:46.846612 22123 tablet_replica.cc:330] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e: stopping tablet replica
I20260811 17:19:46.846762 22123 raft_consensus.cc:2229] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:19:46.846771 21641 tablet_service.cc:1514] Processing DeleteTablet for tablet befb3c92061743a191350097b75114da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:46 UTC) from {username='oryx', principal='oryx/127.9.157.188@KRBTEST.COM'} at 127.0.0.1:54654
I20260811 17:19:46.846973 22123 raft_consensus.cc:2258] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:46.847132 22125 tablet_replica.cc:330] T befb3c92061743a191350097b75114da P 7432ee13358d444494d13841c91bfd5f: stopping tablet replica
I20260811 17:19:46.847436 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 21182
I20260811 17:19:46.847441 22125 raft_consensus.cc:2229] T befb3c92061743a191350097b75114da P 7432ee13358d444494d13841c91bfd5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:19:46.847592 22125 raft_consensus.cc:2258] T befb3c92061743a191350097b75114da P 7432ee13358d444494d13841c91bfd5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:46.850764 22125 ts_tablet_manager.cc:1905] T befb3c92061743a191350097b75114da P 7432ee13358d444494d13841c91bfd5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:46.850541 22123 ts_tablet_manager.cc:1905] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:46.854199 22123 ts_tablet_manager.cc:1918] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:19:46.854327 22123 log.cc:1196] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-2/wal/wals/befb3c92061743a191350097b75114da
I20260811 17:19:46.854663 22123 ts_tablet_manager.cc:1939] T befb3c92061743a191350097b75114da P 4cd592e9c42a4fe8881afa489954dd8e: Deleting consensus metadata
I20260811 17:19:46.854923 22125 ts_tablet_manager.cc:1918] T befb3c92061743a191350097b75114da P 7432ee13358d444494d13841c91bfd5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:19:46.855028 22125 log.cc:1196] T befb3c92061743a191350097b75114da P 7432ee13358d444494d13841c91bfd5f: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786468743878666-9846-0/minicluster-data/ts-1/wal/wals/befb3c92061743a191350097b75114da
I20260811 17:19:46.855453 22125 ts_tablet_manager.cc:1939] T befb3c92061743a191350097b75114da P 7432ee13358d444494d13841c91bfd5f: Deleting consensus metadata
I20260811 17:19:46.856489 21239 catalog_manager.cc:4841] TS 7432ee13358d444494d13841c91bfd5f (127.9.157.130:37833): tablet befb3c92061743a191350097b75114da (table test-table [id=81157c8688a646c4abae67e43462459c]) successfully deleted
I20260811 17:19:46.860582 21238 catalog_manager.cc:4841] TS 4cd592e9c42a4fe8881afa489954dd8e (127.9.157.131:37867): tablet befb3c92061743a191350097b75114da (table test-table [id=81157c8688a646c4abae67e43462459c]) successfully deleted
I20260811 17:19:46.873014 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 21546
I20260811 17:19:46.873348 22093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.872876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:38203 (local address 127.0.0.1:48758)
0811 17:19:46.873124 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:46.873211 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.129:38203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260811 17:19:46.873677 21239 catalog_manager.cc:4584] TS 6756578bcb7b438b8a5bcbaaecc2914a (127.9.157.129:38203): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet befb3c92061743a191350097b75114da: Network error: Client connection negotiation failed: client connection to 127.9.157.129:38203: connect: Connection refused (error 111)
I20260811 17:19:46.892279 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 21761
I20260811 17:19:46.914731 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 20750
I20260811 17:19:46.926671 22093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.926239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:38203 (local address 127.0.0.1:48774)
0811 17:19:46.926453 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:46.926541 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.129:38203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 17:19:46.927556 22093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:46.927284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:38203 (local address 127.0.0.1:48778)
0811 17:19:46.927425 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:46.927475 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.129:38203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 17:19:46.929666 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 20905
2026-08-11T17:19:46Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7913 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:19:47 dist-test-slave-mjjr krb5kdc[22128](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:19:47 dist-test-slave-mjjr krb5kdc[22128](info): set up 2 sockets
Aug 11 17:19:47 dist-test-slave-mjjr krb5kdc[22128](info): commencing operation
krb5kdc: starting...
W20260811 17:19:48.204026 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:19:48 dist-test-slave-mjjr krb5kdc[22128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:19:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:19:48Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.190.keytab.
Entry for principal oryx/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.190.keytab.
I20260811 17:19:48.342921 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:44807
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.190.keytab
--principal=oryx/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:44807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:48.465376 22147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:48.465761 22147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:48.465839 22147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:48.470191 22147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:19:48.470332 22147 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 17:19:48.470368 22147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:48.470400 22147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:48.470433 22147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:19:48.470467 22147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:19:48.476414 22147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:44807
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.190.keytab
--principal=oryx/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:44807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:48.477737 22147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:48.478940 22147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:48.489219 22160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:48.489234 22164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:48.489281 22147 server_base.cc:1034] running on GCE node
W20260811 17:19:48.489229 22157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:48.490257 22147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:48.491868 22147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:48.493208 22147 hybrid_clock.cc:648] HybridClock initialized: now 1786468788493145 us; error 87 us; skew 500 ppm
Aug 11 17:19:48 dist-test-slave-mjjr krb5kdc[22128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468788, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:48.497035 22147 init.cc:377] Logged in from keytab as oryx/127.9.157.190@KRBTEST.COM (short username oryx)
I20260811 17:19:48.498509 22147 webserver.cc:460] Webserver started at http://127.9.157.190:32783/ using document root <none> and password file <none>
I20260811 17:19:48.499204 22147 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:48.499321 22147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:48.499593 22147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:48.502141 22147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "a15553ed49fb4a3d8511c2caf09b8aa6"
format_stamp: "Formatted at 2026-08-11 17:19:48 on dist-test-slave-mjjr"
server_key: "174d1397593d78a4b1ac5cf89cf016a8"
server_key_iv: "fcd4f8c1dd080fa49e8af0d71de9d9e0"
server_key_version: "encryptionkey@0"
I20260811 17:19:48.502807 22147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "a15553ed49fb4a3d8511c2caf09b8aa6"
format_stamp: "Formatted at 2026-08-11 17:19:48 on dist-test-slave-mjjr"
server_key: "174d1397593d78a4b1ac5cf89cf016a8"
server_key_iv: "fcd4f8c1dd080fa49e8af0d71de9d9e0"
server_key_version: "encryptionkey@0"
I20260811 17:19:48.507472 22147 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:19:48.510638 22176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:48.512290 22147 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:48.512486 22147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "a15553ed49fb4a3d8511c2caf09b8aa6"
format_stamp: "Formatted at 2026-08-11 17:19:48 on dist-test-slave-mjjr"
server_key: "174d1397593d78a4b1ac5cf89cf016a8"
server_key_iv: "fcd4f8c1dd080fa49e8af0d71de9d9e0"
server_key_version: "encryptionkey@0"
I20260811 17:19:48.512645 22147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:48.528460 22147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:48.533213 22147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:48.533617 22147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:48.543495 22147 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:44807
I20260811 17:19:48.543571 22282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:44807 every 8 connection(s)
I20260811 17:19:48.544852 22147 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:19:48.549327 22283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:48.552340 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 22147
I20260811 17:19:48.552457 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:19:48.553045 9846 external_mini_cluster.cc:1428] Setting key 3d6739bd7317528e9b8676d2b6da3c82
I20260811 17:19:48.556699 22283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a15553ed49fb4a3d8511c2caf09b8aa6: Bootstrap starting.
I20260811 17:19:48.559826 22283 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a15553ed49fb4a3d8511c2caf09b8aa6: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:48.560920 22283 log.cc:824] T 00000000000000000000000000000000 P a15553ed49fb4a3d8511c2caf09b8aa6: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:48.563578 22283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a15553ed49fb4a3d8511c2caf09b8aa6: No bootstrap required, opened a new log
Aug 11 17:19:48 dist-test-slave-mjjr krb5kdc[22128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM
I20260811 17:19:48.569131 22283 raft_consensus.cc:348] T 00000000000000000000000000000000 P a15553ed49fb4a3d8511c2caf09b8aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a15553ed49fb4a3d8511c2caf09b8aa6" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44807 } }
I20260811 17:19:48.569569 22283 raft_consensus.cc:374] T 00000000000000000000000000000000 P a15553ed49fb4a3d8511c2caf09b8aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:48.569638 22283 raft_consensus.cc:729] T 00000000000000000000000000000000 P a15553ed49fb4a3d8511c2caf09b8aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a15553ed49fb4a3d8511c2caf09b8aa6, State: Initialized, Role: FOLLOWER
I20260811 17:19:48.570314 22283 consensus_queue.cc:260] T 00000000000000000000000000000000 P a15553ed49fb4a3d8511c2caf09b8aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a15553ed49fb4a3d8511c2caf09b8aa6" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44807 } }
I20260811 17:19:48.570525 22283 raft_consensus.cc:388] T 00000000000000000000000000000000 P a15553ed49fb4a3d8511c2caf09b8aa6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:19:48.570601 22283 raft_consensus.cc:482] T 00000000000000000000000000000000 P a15553ed49fb4a3d8511c2caf09b8aa6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:19:48.570714 22283 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a15553ed49fb4a3d8511c2caf09b8aa6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:48.571719 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:48.555077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32996 (local address 127.9.157.190:44807)
0811 17:19:48.555620 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:48.555696 (+ 76us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:48.555753 (+ 57us) server_negotiation.cc:407] Connection header received
0811 17:19:48.556505 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:48.556524 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:48.556942 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:48.557273 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:48.558354 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:48.559796 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:48.561163 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:48.561699 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:48.561987 (+ 288us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:48.564739 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:48.564746 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:48.564767 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:48.564819 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:48.567339 (+ 2520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:48.568079 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:48.568089 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:48.568097 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:48.568283 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:48.568644 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:48.568648 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:48.568650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:48.569312 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:48.569635 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:48.570052 (+ 417us) server_negotiation.cc:299] Negotiation successful
0811 17:19:48.570437 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":189,"threads_started":1}
I20260811 17:19:48.572309 22283 raft_consensus.cc:504] T 00000000000000000000000000000000 P a15553ed49fb4a3d8511c2caf09b8aa6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a15553ed49fb4a3d8511c2caf09b8aa6" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44807 } }
I20260811 17:19:48.572782 22283 leader_election.cc:302] T 00000000000000000000000000000000 P a15553ed49fb4a3d8511c2caf09b8aa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a15553ed49fb4a3d8511c2caf09b8aa6; no voters:
I20260811 17:19:48.573238 22283 leader_election.cc:288] T 00000000000000000000000000000000 P a15553ed49fb4a3d8511c2caf09b8aa6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:19:48.573380 22292 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a15553ed49fb4a3d8511c2caf09b8aa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:48.573659 22292 raft_consensus.cc:686] T 00000000000000000000000000000000 P a15553ed49fb4a3d8511c2caf09b8aa6 [term 1 LEADER]: Becoming Leader. State: Replica: a15553ed49fb4a3d8511c2caf09b8aa6, State: Running, Role: LEADER
I20260811 17:19:48.574122 22292 consensus_queue.cc:237] T 00000000000000000000000000000000 P a15553ed49fb4a3d8511c2caf09b8aa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a15553ed49fb4a3d8511c2caf09b8aa6" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44807 } }
I20260811 17:19:48.574888 22283 sys_catalog.cc:564] T 00000000000000000000000000000000 P a15553ed49fb4a3d8511c2caf09b8aa6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:19:48.575726 22292 sys_catalog.cc:455] T 00000000000000000000000000000000 P a15553ed49fb4a3d8511c2caf09b8aa6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a15553ed49fb4a3d8511c2caf09b8aa6. Latest consensus state: current_term: 1 leader_uuid: "a15553ed49fb4a3d8511c2caf09b8aa6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a15553ed49fb4a3d8511c2caf09b8aa6" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44807 } } }
I20260811 17:19:48.575981 22292 sys_catalog.cc:458] T 00000000000000000000000000000000 P a15553ed49fb4a3d8511c2caf09b8aa6 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:48.576226 22293 sys_catalog.cc:455] T 00000000000000000000000000000000 P a15553ed49fb4a3d8511c2caf09b8aa6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a15553ed49fb4a3d8511c2caf09b8aa6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a15553ed49fb4a3d8511c2caf09b8aa6" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44807 } } }
I20260811 17:19:48.576433 22293 sys_catalog.cc:458] T 00000000000000000000000000000000 P a15553ed49fb4a3d8511c2caf09b8aa6 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:48.576508 22302 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:19:48.580526 22302 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:19:48.588459 22302 catalog_manager.cc:1347] Generated new cluster ID: 71f915b76a56419fa44750f054020083
I20260811 17:19:48.588689 22302 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:19:48.605937 22302 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:19:48.607218 22302 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:19:48.614606 22302 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a15553ed49fb4a3d8511c2caf09b8aa6: Generated new TSK 0
I20260811 17:19:48.615481 22302 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.129.keytab.
Entry for principal oryx/127.9.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.129.keytab.
I20260811 17:19:48.658700 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:0
--local_ip_for_outbound_sockets=127.9.157.129
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:44807
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.129.keytab
--principal=oryx/127.9.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:48.795303 22326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:48.795799 22326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:48.795933 22326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:48.800362 22326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:48.800520 22326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:48.800625 22326 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 17:19:48.800734 22326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:19:48.807407 22326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.129.keytab
--principal=oryx/127.9.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.157.190:44807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:48.809303 22326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:48.810670 22326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:48.822551 22334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:48.822525 22340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:48.822518 22335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:48.822911 22326 server_base.cc:1034] running on GCE node
I20260811 17:19:48.823462 22326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:48.824342 22326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:48.825564 22326 hybrid_clock.cc:648] HybridClock initialized: now 1786468788825520 us; error 49 us; skew 500 ppm
Aug 11 17:19:48 dist-test-slave-mjjr krb5kdc[22128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468788, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:48.829016 22326 init.cc:377] Logged in from keytab as oryx/127.9.157.129@KRBTEST.COM (short username oryx)
I20260811 17:19:48.830142 22326 webserver.cc:460] Webserver started at http://127.9.157.129:36205/ using document root <none> and password file <none>
I20260811 17:19:48.830758 22326 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:48.830826 22326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:48.831033 22326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:48.833413 22326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-0/data/instance:
uuid: "1096fd23ccb24c7183fc39f31a1e00fc"
format_stamp: "Formatted at 2026-08-11 17:19:48 on dist-test-slave-mjjr"
server_key: "b65c34413768ee1eb33723b82c18e8d1"
server_key_iv: "b3a91c3d323340f9fbeec13b5fe772f2"
server_key_version: "encryptionkey@0"
I20260811 17:19:48.834018 22326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance:
uuid: "1096fd23ccb24c7183fc39f31a1e00fc"
format_stamp: "Formatted at 2026-08-11 17:19:48 on dist-test-slave-mjjr"
server_key: "b65c34413768ee1eb33723b82c18e8d1"
server_key_iv: "b3a91c3d323340f9fbeec13b5fe772f2"
server_key_version: "encryptionkey@0"
I20260811 17:19:48.839046 22326 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 17:19:48.842344 22349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:48.844092 22326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:48.844280 22326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "1096fd23ccb24c7183fc39f31a1e00fc"
format_stamp: "Formatted at 2026-08-11 17:19:48 on dist-test-slave-mjjr"
server_key: "b65c34413768ee1eb33723b82c18e8d1"
server_key_iv: "b3a91c3d323340f9fbeec13b5fe772f2"
server_key_version: "encryptionkey@0"
I20260811 17:19:48.844444 22326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:48.863086 22326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:48.867524 22326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:48.867877 22326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:48.869710 22326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:48.869771 22326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:48.869815 22326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:48.869874 22326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:48.882906 22326 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:45711
I20260811 17:19:48.883060 22547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:45711 every 8 connection(s)
Aug 11 17:19:48 dist-test-slave-mjjr krb5kdc[22128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468788, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM
I20260811 17:19:48.884244 22326 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:19:48.889010 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:48.870985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:57603 (local address 127.9.157.190:44807)
0811 17:19:48.871137 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:48.871141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:48.872780 (+ 1639us) server_negotiation.cc:407] Connection header received
0811 17:19:48.874945 (+ 2165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:48.874950 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:48.875025 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:48.875147 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:48.877718 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:48.878694 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:48.880428 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:48.880874 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:48.881000 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:48.884736 (+ 3736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:48.884740 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:48.884742 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:48.884777 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:48.886514 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:48.887159 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:48.887163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:48.887164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:48.887251 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:48.887699 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:48.887707 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:48.887708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:48.887931 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:48.888059 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:48.888658 (+ 599us) server_negotiation.cc:299] Negotiation successful
0811 17:19:48.888822 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 17:19:48.889995 22364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:48.871616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44807 (local address 127.9.157.129:57603)
0811 17:19:48.872429 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:48.872556 (+ 127us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:48.874372 (+ 1816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:48.875465 (+ 1093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:48.875477 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:48.876228 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:48.877462 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:48.877490 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:48.878893 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:48.878900 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:48.880201 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:48.880215 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:48.881121 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:48.881128 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:48.881492 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:48.882308 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:48.882337 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:48.884462 (+ 2125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:48.886668 (+ 2206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:48.886675 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:48.886694 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:48.887017 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:48.887360 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:48.887364 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:48.887367 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:48.887536 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:48.888058 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:48.888068 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:48.888379 (+ 311us) client_negotiation.cc:769] Sending connection context
0811 17:19:48.888593 (+ 214us) client_negotiation.cc:240] Negotiation successful
0811 17:19:48.889007 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":155,"threads_started":1}
I20260811 17:19:48.891213 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 22326
I20260811 17:19:48.891359 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance
I20260811 17:19:48.891670 22548 heartbeater.cc:347] Connected to a master server at 127.9.157.190:44807
I20260811 17:19:48.891806 9846 external_mini_cluster.cc:1428] Setting key 9c761e6b1d42c434991d09920632c2fb
I20260811 17:19:48.891897 22548 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:48.892642 22548 heartbeater.cc:511] Master 127.9.157.190:44807 requested a full tablet report, sending...
I20260811 17:19:48.894546 22206 ts_manager.cc:194] Registered new tserver with Master: 1096fd23ccb24c7183fc39f31a1e00fc (127.9.157.129:45711)
I20260811 17:19:48.896281 22206 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.157.129@KRBTEST.COM'} at 127.9.157.129:57603
I20260811 17:19:48.903821 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:48.894419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:54205 (local address 127.9.157.190:44807)
0811 17:19:48.894560 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:48.894565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:48.895012 (+ 447us) server_negotiation.cc:407] Connection header received
0811 17:19:48.895224 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:48.895230 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:48.895278 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:48.895332 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:48.896026 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:48.896654 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:48.897738 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:48.898075 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:48.898154 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:48.899719 (+ 1565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:48.899725 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:48.899733 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:48.899757 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:48.901575 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:48.902396 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:48.902400 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:48.902402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:48.902524 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:48.902829 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:48.902833 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:48.902835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:48.903053 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:48.903176 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:48.903461 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 17:19:48.903589 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":75,"server-negotiator.queue_time_us":58}
I20260811 17:19:48.903827 22557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:48.894491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44807 (local address 127.9.157.129:54205)
0811 17:19:48.894868 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:48.894884 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:48.895100 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:48.895448 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:48.895450 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:48.895742 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:48.895880 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:48.895887 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:48.896786 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:48.896790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:48.897589 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:48.897601 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:48.898329 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:48.898333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:48.898426 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:48.899082 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:48.899097 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:48.899541 (+ 444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:48.902016 (+ 2475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:48.902021 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:48.902024 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:48.902275 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:48.902632 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:48.902635 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:48.902637 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:48.902733 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:48.903163 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:48.903167 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:48.903283 (+ 116us) client_negotiation.cc:769] Sending connection context
0811 17:19:48.903419 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 17:19:48.903598 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":116,"threads_started":1}
WARNING: no policy specified for oryx/127.9.157.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.157.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.130.keytab.
Entry for principal oryx/127.9.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.130.keytab.
I20260811 17:19:48.926836 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:0
--local_ip_for_outbound_sockets=127.9.157.130
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:44807
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.130.keytab
--principal=oryx/127.9.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:49.055233 22561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:49.055593 22561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:49.055711 22561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:49.060140 22561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:49.060272 22561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:49.060352 22561 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 17:19:49.060441 22561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:19:49.066533 22561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.130.keytab
--principal=oryx/127.9.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.157.190:44807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:49.067829 22561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:49.068948 22561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:49.079360 22569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:49.079463 22572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:49.079362 22568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:49.079613 22561 server_base.cc:1034] running on GCE node
I20260811 17:19:49.080206 22561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:49.081022 22561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:49.082237 22561 hybrid_clock.cc:648] HybridClock initialized: now 1786468789082216 us; error 37 us; skew 500 ppm
Aug 11 17:19:49 dist-test-slave-mjjr krb5kdc[22128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468789, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:49.085501 22561 init.cc:377] Logged in from keytab as oryx/127.9.157.130@KRBTEST.COM (short username oryx)
I20260811 17:19:49.086505 22561 webserver.cc:460] Webserver started at http://127.9.157.130:41249/ using document root <none> and password file <none>
I20260811 17:19:49.087299 22561 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:49.087404 22561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:49.087718 22561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:49.091018 22561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-1/data/instance:
uuid: "9fbf9ebada614af9b5679b3e602cecd8"
format_stamp: "Formatted at 2026-08-11 17:19:49 on dist-test-slave-mjjr"
server_key: "816907d9036781939868fa524d94ab21"
server_key_iv: "b885e7b59398dfaff19f97fed8dbe8ca"
server_key_version: "encryptionkey@0"
I20260811 17:19:49.091928 22561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance:
uuid: "9fbf9ebada614af9b5679b3e602cecd8"
format_stamp: "Formatted at 2026-08-11 17:19:49 on dist-test-slave-mjjr"
server_key: "816907d9036781939868fa524d94ab21"
server_key_iv: "b885e7b59398dfaff19f97fed8dbe8ca"
server_key_version: "encryptionkey@0"
I20260811 17:19:49.097669 22561 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 17:19:49.100630 22581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:49.102073 22561 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:49.102244 22561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "9fbf9ebada614af9b5679b3e602cecd8"
format_stamp: "Formatted at 2026-08-11 17:19:49 on dist-test-slave-mjjr"
server_key: "816907d9036781939868fa524d94ab21"
server_key_iv: "b885e7b59398dfaff19f97fed8dbe8ca"
server_key_version: "encryptionkey@0"
I20260811 17:19:49.102389 22561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:49.119095 22561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:49.122800 22561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:49.123056 22561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:49.124790 22561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:49.124872 22561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:49.124949 22561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:49.125001 22561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 17:19:49 dist-test-slave-mjjr krb5kdc[22128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468789, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM
I20260811 17:19:49.136564 22561 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:40473
I20260811 17:19:49.136636 22744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:40473 every 8 connection(s)
I20260811 17:19:49.137669 22561 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:19:49.140115 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.125735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:60085 (local address 127.9.157.190:44807)
0811 17:19:49.125876 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:49.125881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:49.126819 (+ 938us) server_negotiation.cc:407] Connection header received
0811 17:19:49.127887 (+ 1068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:49.127890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:49.127968 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:49.128015 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:49.129475 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.130396 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.132416 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.132842 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.132969 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.136224 (+ 3255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:49.136228 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:49.136231 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:49.136261 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:49.137757 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:49.138440 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:49.138444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:49.138446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:49.138534 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:49.138928 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:49.138931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:49.138933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:49.139141 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:49.139265 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:49.139811 (+ 546us) server_negotiation.cc:299] Negotiation successful
0811 17:19:49.139941 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 17:19:49.140949 22593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.126147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44807 (local address 127.9.157.130:60085)
0811 17:19:49.126638 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:49.126693 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:49.127612 (+ 919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:49.128407 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:49.128414 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:49.128730 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:49.129214 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:49.129252 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.130668 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.130674 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:49.132257 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:49.132271 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.133037 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.133041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:49.133201 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.133861 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:49.133878 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:49.135992 (+ 2114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:49.137903 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:49.137909 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:49.137924 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:49.138303 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:49.138634 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:49.138637 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:49.138640 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:49.138834 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:49.139256 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:49.139263 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:49.139522 (+ 259us) client_negotiation.cc:769] Sending connection context
0811 17:19:49.139759 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 17:19:49.140094 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":134,"threads_started":1}
I20260811 17:19:49.142159 22745 heartbeater.cc:347] Connected to a master server at 127.9.157.190:44807
I20260811 17:19:49.142309 22745 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:49.142908 22745 heartbeater.cc:511] Master 127.9.157.190:44807 requested a full tablet report, sending...
I20260811 17:19:49.144270 22203 ts_manager.cc:194] Registered new tserver with Master: 9fbf9ebada614af9b5679b3e602cecd8 (127.9.157.130:40473)
I20260811 17:19:49.145102 22203 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.157.130@KRBTEST.COM'} at 127.9.157.130:60085
I20260811 17:19:49.146270 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 22561
I20260811 17:19:49.146394 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance
I20260811 17:19:49.146742 9846 external_mini_cluster.cc:1428] Setting key ab432df3294dabb9b242d07867be810b
I20260811 17:19:49.152530 22754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.144693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44807 (local address 127.9.157.130:48411)
0811 17:19:49.145014 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:49.145026 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:49.145143 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:49.145390 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:49.145392 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:49.145613 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:49.145746 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:49.145754 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.146763 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.146767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:49.147509 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:49.147518 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.148015 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.148019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:49.148129 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.148692 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:49.148703 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:49.149092 (+ 389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:49.150871 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:49.150874 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:49.150875 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:49.151093 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:49.151409 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:49.151411 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:49.151413 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:49.151489 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:49.152083 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:49.152086 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:49.152172 (+ 86us) client_negotiation.cc:769] Sending connection context
0811 17:19:49.152263 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 17:19:49.152382 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":121,"threads_started":1}
I20260811 17:19:49.152730 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.144748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:48411 (local address 127.9.157.190:44807)
0811 17:19:49.144919 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:49.144924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:49.145100 (+ 176us) server_negotiation.cc:407] Connection header received
0811 17:19:49.145235 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:49.145238 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:49.145287 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:49.145374 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:49.145872 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.146618 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.147636 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.147884 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.148010 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.149234 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:49.149242 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:49.149244 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:49.149267 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:49.150733 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:49.151203 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:49.151208 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:49.151210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:49.151308 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:49.151584 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:49.151589 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:49.151592 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:49.151974 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:49.152087 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:49.152354 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 17:19:49.152492 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
WARNING: no policy specified for oryx/127.9.157.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.157.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.131.keytab.
Entry for principal oryx/127.9.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.131.keytab.
I20260811 17:19:49.176935 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:0
--local_ip_for_outbound_sockets=127.9.157.131
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:44807
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.131.keytab
--principal=oryx/127.9.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:49.297533 22759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:49.297848 22759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:49.297897 22759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:49.302168 22759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:49.302250 22759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:49.302294 22759 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 17:19:49.302366 22759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:19:49.308209 22759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.131.keytab
--principal=oryx/127.9.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.157.190:44807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:49.309486 22759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:49.310582 22759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:49.320938 22772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:49.321069 22769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:49.320938 22767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:49.321230 22759 server_base.cc:1034] running on GCE node
I20260811 17:19:49.321784 22759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:49.322486 22759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:49.323721 22759 hybrid_clock.cc:648] HybridClock initialized: now 1786468789323704 us; error 34 us; skew 500 ppm
Aug 11 17:19:49 dist-test-slave-mjjr krb5kdc[22128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468789, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:49.326603 22759 init.cc:377] Logged in from keytab as oryx/127.9.157.131@KRBTEST.COM (short username oryx)
I20260811 17:19:49.327581 22759 webserver.cc:460] Webserver started at http://127.9.157.131:38981/ using document root <none> and password file <none>
I20260811 17:19:49.328219 22759 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:49.328315 22759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:49.328568 22759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:49.330792 22759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-2/data/instance:
uuid: "7c711e5e8faf44e8bc7a9138e155773c"
format_stamp: "Formatted at 2026-08-11 17:19:49 on dist-test-slave-mjjr"
server_key: "d7f9a5612837ae2de976c1583766fcfc"
server_key_iv: "9832a1af81ac33adf154d4a491cafc4a"
server_key_version: "encryptionkey@0"
I20260811 17:19:49.331401 22759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance:
uuid: "7c711e5e8faf44e8bc7a9138e155773c"
format_stamp: "Formatted at 2026-08-11 17:19:49 on dist-test-slave-mjjr"
server_key: "d7f9a5612837ae2de976c1583766fcfc"
server_key_iv: "9832a1af81ac33adf154d4a491cafc4a"
server_key_version: "encryptionkey@0"
I20260811 17:19:49.335554 22759 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 17:19:49.338284 22783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:49.339738 22759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:49.339876 22759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "7c711e5e8faf44e8bc7a9138e155773c"
format_stamp: "Formatted at 2026-08-11 17:19:49 on dist-test-slave-mjjr"
server_key: "d7f9a5612837ae2de976c1583766fcfc"
server_key_iv: "9832a1af81ac33adf154d4a491cafc4a"
server_key_version: "encryptionkey@0"
I20260811 17:19:49.340018 22759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:49.365401 22759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:49.369057 22759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:49.369323 22759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:49.371052 22759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:49.371137 22759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:49.371217 22759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:49.371260 22759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 17:19:49 dist-test-slave-mjjr krb5kdc[22128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468789, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM
I20260811 17:19:49.384248 22759 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:46347
I20260811 17:19:49.384323 22948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:46347 every 8 connection(s)
I20260811 17:19:49.385547 22759 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:19:49.387324 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.372047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:35371 (local address 127.9.157.190:44807)
0811 17:19:49.372192 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:49.372199 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:49.373426 (+ 1227us) server_negotiation.cc:407] Connection header received
0811 17:19:49.374862 (+ 1436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:49.374866 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:49.374956 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:49.375081 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:49.376782 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.377515 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.378867 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.379107 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.379240 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.382278 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:49.382286 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:49.382289 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:49.382331 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:49.384279 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:49.385035 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:49.385038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:49.385040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:49.385130 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:49.385578 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:49.385580 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:49.385581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:49.385855 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:49.385944 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:49.386835 (+ 891us) server_negotiation.cc:299] Negotiation successful
0811 17:19:49.387143 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 17:19:49.388299 22796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.372668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44807 (local address 127.9.157.131:35371)
0811 17:19:49.373199 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:49.373266 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:49.374521 (+ 1255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:49.375280 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:49.375290 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:49.375923 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:49.376610 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:49.376626 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.377654 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.377658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:49.378717 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:49.378737 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.379293 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.379297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:49.379528 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.380302 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:49.380320 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:49.382045 (+ 1725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:49.384450 (+ 2405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:49.384456 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:49.384473 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:49.384825 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:49.385248 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:49.385251 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:49.385253 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:49.385469 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:49.385975 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:49.385982 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:49.386352 (+ 370us) client_negotiation.cc:769] Sending connection context
0811 17:19:49.386779 (+ 427us) client_negotiation.cc:240] Negotiation successful
0811 17:19:49.387306 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":110,"threads_started":1}
I20260811 17:19:49.389734 22950 heartbeater.cc:347] Connected to a master server at 127.9.157.190:44807
I20260811 17:19:49.389961 22950 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:49.390683 22950 heartbeater.cc:511] Master 127.9.157.190:44807 requested a full tablet report, sending...
I20260811 17:19:49.392241 22206 ts_manager.cc:194] Registered new tserver with Master: 7c711e5e8faf44e8bc7a9138e155773c (127.9.157.131:46347)
I20260811 17:19:49.393172 22206 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.157.131@KRBTEST.COM'} at 127.9.157.131:35371
I20260811 17:19:49.395895 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 22759
I20260811 17:19:49.396068 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance
I20260811 17:19:49.396404 9846 external_mini_cluster.cc:1428] Setting key fdd38f4b021d8407c35ceb721d4cd6d6
I20260811 17:19:49.398866 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:19:49.400839 22960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.392752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44807 (local address 127.9.157.131:48497)
0811 17:19:49.393135 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:49.393149 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:49.393279 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:49.393566 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:49.393569 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:49.393829 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:49.393995 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:49.394004 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.394838 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.394841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:49.395633 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:49.395685 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.396205 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.396209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:49.396306 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.396919 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:49.396932 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:49.397376 (+ 444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:49.399231 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:49.399234 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:49.399235 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:49.399437 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:49.399799 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:49.399802 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:49.399803 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:49.399885 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:49.400364 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:49.400366 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:49.400452 (+ 86us) client_negotiation.cc:769] Sending connection context
0811 17:19:49.400535 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 17:19:49.400658 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":4,"thread_start_us":124,"threads_started":1}
I20260811 17:19:49.400986 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.392855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:48497 (local address 127.9.157.190:44807)
0811 17:19:49.393030 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:49.393035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:49.393227 (+ 192us) server_negotiation.cc:407] Connection header received
0811 17:19:49.393400 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:49.393403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:49.393462 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:49.393547 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:49.394135 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.394652 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.395805 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.396075 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.396196 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.397509 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:49.397513 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:49.397515 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:49.397542 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:49.399120 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:49.399542 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:49.399546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:49.399548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:49.399681 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:49.399984 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:49.399990 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:49.399994 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:49.400262 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:49.400375 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:49.400623 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 17:19:49.400762 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
Aug 11 17:19:49 dist-test-slave-mjjr krb5kdc[22128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468789, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 17:19:49 dist-test-slave-mjjr krb5kdc[22128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:19:49 dist-test-slave-mjjr krb5kdc[22128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 17:19:49.426671 22288 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:49.420233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33006 (local address 127.9.157.190:44807)
0811 17:19:49.420377 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:49.420381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:49.420393 (+ 12us) server_negotiation.cc:407] Connection header received
0811 17:19:49.420436 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:49.420438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:49.420477 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:49.420544 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:49.421341 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.421856 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.422722 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.422929 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.423001 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.426533 (+ 3532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33006: BlockingRecv error: recv got EOF from 127.0.0.1:33006 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 17:19:49 dist-test-slave-mjjr krb5kdc[22128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468789, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM
I20260811 17:19:49.436939 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.428445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33014 (local address 127.9.157.190:44807)
0811 17:19:49.428611 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:49.428616 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:49.428774 (+ 158us) server_negotiation.cc:407] Connection header received
0811 17:19:49.428943 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:49.428946 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:49.428999 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:49.429092 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:49.429719 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.430248 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.431052 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.431288 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.431402 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.433855 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:49.433859 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:49.433861 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:49.433886 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:49.435198 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:49.435635 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:49.435725 (+ 90us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:49.435727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:49.435831 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:49.436104 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:49.436107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:49.436108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:49.436305 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:49.436422 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:49.436673 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 17:19:49.436812 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 17:19:49.440048 22206 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33014:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:19:49.443603 22206 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:19:49.459621 22984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.452975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:45711 (local address 127.0.0.1:48252)
0811 17:19:49.454206 (+ 1231us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:49.454216 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:49.454283 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:49.454832 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:49.454835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:49.454854 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:49.454959 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:49.454965 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.456631 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.456635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:49.457816 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:49.457827 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.458809 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.458812 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:49.459013 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.459097 (+ 84us) client_negotiation.cc:769] Sending connection context
0811 17:19:49.459219 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 17:19:49.459410 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1184,"mutex_wait_us":180,"thread_start_us":96,"threads_started":1}
I20260811 17:19:49.459844 22986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.452761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56196 (local address 127.9.157.130:40473)
0811 17:19:49.454546 (+ 1785us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:49.454560 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:49.454581 (+ 21us) server_negotiation.cc:407] Connection header received
0811 17:19:49.454653 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:49.454657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:49.454863 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:49.455023 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:49.455218 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.456035 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.457834 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.458676 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.458802 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.458982 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:49.459275 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 17:19:49.459664 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1657,"thread_start_us":145,"threads_started":1}
I20260811 17:19:49.459621 22982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.452601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:40473 (local address 127.0.0.1:56196)
0811 17:19:49.453337 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:49.453390 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:49.453527 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:49.455001 (+ 1474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:49.455004 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:49.455022 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:49.455103 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:49.455109 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.456191 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.456197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:49.457531 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:49.457541 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.458809 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.458812 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:49.459018 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.459097 (+ 79us) client_negotiation.cc:769] Sending connection context
0811 17:19:49.459219 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 17:19:49.459395 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":154,"thread_start_us":89,"threads_started":1}
I20260811 17:19:49.460335 22989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.453427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48252 (local address 127.9.157.129:45711)
0811 17:19:49.454208 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:49.454214 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:49.454284 (+ 70us) server_negotiation.cc:407] Connection header received
0811 17:19:49.454376 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:49.454380 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:49.454560 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:49.454688 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:49.455094 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.456500 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.457946 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.458678 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.458800 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.458957 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:49.460083 (+ 1126us) server_negotiation.cc:299] Negotiation successful
0811 17:19:49.460191 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"thread_start_us":97,"threads_started":1}
I20260811 17:19:49.461090 22987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.452675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:46347 (local address 127.0.0.1:34336)
0811 17:19:49.453887 (+ 1212us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:49.453900 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:49.454004 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:49.454563 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:49.454571 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:49.454598 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:49.454741 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:49.454747 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.456273 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.456278 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:49.457816 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:49.457827 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.460646 (+ 2819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.460649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:49.460742 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.460766 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 17:19:49.460837 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 17:19:49.460958 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1144,"mutex_wait_us":80,"thread_start_us":84,"threads_started":1}
I20260811 17:19:49.461591 22990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.453450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34336 (local address 127.9.157.131:46347)
0811 17:19:49.453956 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:49.453970 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:49.453997 (+ 27us) server_negotiation.cc:407] Connection header received
0811 17:19:49.454159 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:49.454165 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:49.454386 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:49.454551 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:49.454868 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.456125 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.459030 (+ 2905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.460292 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.460404 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.460581 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:49.461357 (+ 776us) server_negotiation.cc:299] Negotiation successful
0811 17:19:49.461456 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":122,"threads_started":1}
I20260811 17:19:49.463240 22434 tablet_service.cc:1467] Processing CreateTablet for tablet 9de18c9eb3954af99611b2bda8871fe8 (DEFAULT_TABLE table=test-table [id=849db498d63e44c8afba49acb5ee8482]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:49.463285 22652 tablet_service.cc:1467] Processing CreateTablet for tablet 9de18c9eb3954af99611b2bda8871fe8 (DEFAULT_TABLE table=test-table [id=849db498d63e44c8afba49acb5ee8482]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:49.464401 22434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9de18c9eb3954af99611b2bda8871fe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:49.464440 22652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9de18c9eb3954af99611b2bda8871fe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:49.464228 22851 tablet_service.cc:1467] Processing CreateTablet for tablet 9de18c9eb3954af99611b2bda8871fe8 (DEFAULT_TABLE table=test-table [id=849db498d63e44c8afba49acb5ee8482]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:49.465179 22851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9de18c9eb3954af99611b2bda8871fe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:49.470817 22991 tablet_bootstrap.cc:492] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc: Bootstrap starting.
I20260811 17:19:49.472869 22995 tablet_bootstrap.cc:492] T 9de18c9eb3954af99611b2bda8871fe8 P 9fbf9ebada614af9b5679b3e602cecd8: Bootstrap starting.
I20260811 17:19:49.473325 22993 tablet_bootstrap.cc:492] T 9de18c9eb3954af99611b2bda8871fe8 P 7c711e5e8faf44e8bc7a9138e155773c: Bootstrap starting.
I20260811 17:19:49.474871 22991 tablet_bootstrap.cc:654] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:49.475631 22993 tablet_bootstrap.cc:654] T 9de18c9eb3954af99611b2bda8871fe8 P 7c711e5e8faf44e8bc7a9138e155773c: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:49.475838 22991 log.cc:824] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:49.476516 22993 log.cc:824] T 9de18c9eb3954af99611b2bda8871fe8 P 7c711e5e8faf44e8bc7a9138e155773c: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:49.477784 22995 tablet_bootstrap.cc:654] T 9de18c9eb3954af99611b2bda8871fe8 P 9fbf9ebada614af9b5679b3e602cecd8: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:49.478369 22993 tablet_bootstrap.cc:492] T 9de18c9eb3954af99611b2bda8871fe8 P 7c711e5e8faf44e8bc7a9138e155773c: No bootstrap required, opened a new log
I20260811 17:19:49.478401 22991 tablet_bootstrap.cc:492] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc: No bootstrap required, opened a new log
I20260811 17:19:49.478612 22991 ts_tablet_manager.cc:1397] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 17:19:49.478591 22995 log.cc:824] T 9de18c9eb3954af99611b2bda8871fe8 P 9fbf9ebada614af9b5679b3e602cecd8: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:49.478649 22993 ts_tablet_manager.cc:1397] T 9de18c9eb3954af99611b2bda8871fe8 P 7c711e5e8faf44e8bc7a9138e155773c: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260811 17:19:49.481264 22995 tablet_bootstrap.cc:492] T 9de18c9eb3954af99611b2bda8871fe8 P 9fbf9ebada614af9b5679b3e602cecd8: No bootstrap required, opened a new log
I20260811 17:19:49.481491 22995 ts_tablet_manager.cc:1397] T 9de18c9eb3954af99611b2bda8871fe8 P 9fbf9ebada614af9b5679b3e602cecd8: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260811 17:19:49.482115 22993 raft_consensus.cc:348] T 9de18c9eb3954af99611b2bda8871fe8 P 7c711e5e8faf44e8bc7a9138e155773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } } peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } }
I20260811 17:19:49.482372 22993 raft_consensus.cc:374] T 9de18c9eb3954af99611b2bda8871fe8 P 7c711e5e8faf44e8bc7a9138e155773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:49.482460 22993 raft_consensus.cc:729] T 9de18c9eb3954af99611b2bda8871fe8 P 7c711e5e8faf44e8bc7a9138e155773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c711e5e8faf44e8bc7a9138e155773c, State: Initialized, Role: FOLLOWER
I20260811 17:19:49.482954 22993 consensus_queue.cc:260] T 9de18c9eb3954af99611b2bda8871fe8 P 7c711e5e8faf44e8bc7a9138e155773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } } peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } }
I20260811 17:19:49.483362 22991 raft_consensus.cc:348] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } } peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } }
I20260811 17:19:49.483706 22991 raft_consensus.cc:374] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:49.483706 22950 heartbeater.cc:503] Master 127.9.157.190:44807 was elected leader, sending a full tablet report...
I20260811 17:19:49.483814 22991 raft_consensus.cc:729] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1096fd23ccb24c7183fc39f31a1e00fc, State: Initialized, Role: FOLLOWER
I20260811 17:19:49.484098 22993 ts_tablet_manager.cc:1428] T 9de18c9eb3954af99611b2bda8871fe8 P 7c711e5e8faf44e8bc7a9138e155773c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 17:19:49.484360 22991 consensus_queue.cc:260] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } } peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } }
I20260811 17:19:49.485040 22548 heartbeater.cc:503] Master 127.9.157.190:44807 was elected leader, sending a full tablet report...
I20260811 17:19:49.485311 22991 ts_tablet_manager.cc:1428] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc: Time spent starting tablet: real 0.007s user 0.000s sys 0.006s
I20260811 17:19:49.485486 22995 raft_consensus.cc:348] T 9de18c9eb3954af99611b2bda8871fe8 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } } peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } }
I20260811 17:19:49.485746 22995 raft_consensus.cc:374] T 9de18c9eb3954af99611b2bda8871fe8 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:49.486093 22995 raft_consensus.cc:729] T 9de18c9eb3954af99611b2bda8871fe8 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fbf9ebada614af9b5679b3e602cecd8, State: Initialized, Role: FOLLOWER
I20260811 17:19:49.486598 22995 consensus_queue.cc:260] T 9de18c9eb3954af99611b2bda8871fe8 P 9fbf9ebada614af9b5679b3e602cecd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } } peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } }
I20260811 17:19:49.487538 22745 heartbeater.cc:503] Master 127.9.157.190:44807 was elected leader, sending a full tablet report...
I20260811 17:19:49.487736 22995 ts_tablet_manager.cc:1428] T 9de18c9eb3954af99611b2bda8871fe8 P 9fbf9ebada614af9b5679b3e602cecd8: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260811 17:19:49.512639 23004 raft_consensus.cc:482] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:19:49.512940 23004 raft_consensus.cc:504] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } } peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } }
I20260811 17:19:49.514202 23004 leader_election.cc:288] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9fbf9ebada614af9b5679b3e602cecd8 (127.9.157.130:40473), 7c711e5e8faf44e8bc7a9138e155773c (127.9.157.131:46347)
I20260811 17:19:49.518839 22986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.514642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:50363 (local address 127.9.157.130:40473)
0811 17:19:49.514813 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:49.514820 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:49.515062 (+ 242us) server_negotiation.cc:407] Connection header received
0811 17:19:49.515243 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:49.515247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:49.515289 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:49.515375 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:49.515689 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.516233 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.517516 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.518189 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.518317 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.518397 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:49.518619 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 17:19:49.518701 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 17:19:49.518839 22990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.514627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:52633 (local address 127.9.157.131:46347)
0811 17:19:49.514852 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:49.514858 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:49.515148 (+ 290us) server_negotiation.cc:407] Connection header received
0811 17:19:49.515319 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:49.515323 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:49.515373 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:49.515462 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:49.515809 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.516387 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.517719 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.518311 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.518440 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.518491 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:49.518644 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 17:19:49.518714 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260811 17:19:49.518862 23008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.514500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:40473 (local address 127.9.157.129:50363)
0811 17:19:49.514955 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:49.514968 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:49.515113 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:49.515389 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:49.515392 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:49.515418 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:49.515548 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:49.515553 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.516349 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.516355 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:49.517385 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:49.517395 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.518318 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.518328 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:49.518453 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.518484 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 17:19:49.518570 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 17:19:49.518658 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":149,"threads_started":1}
I20260811 17:19:49.518873 23011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.514500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:46347 (local address 127.9.157.129:52633)
0811 17:19:49.515042 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:49.515054 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:49.515190 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:49.515475 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:49.515478 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:49.515495 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:49.515628 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:49.515633 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.516500 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.516507 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:49.517589 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:49.517599 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.518420 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.518423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:49.518516 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.518547 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 17:19:49.518616 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 17:19:49.518660 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"mutex_wait_us":48,"thread_start_us":140,"threads_started":1}
I20260811 17:19:49.519887 22656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9de18c9eb3954af99611b2bda8871fe8" candidate_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fbf9ebada614af9b5679b3e602cecd8" is_pre_election: true
I20260811 17:19:49.519887 22877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9de18c9eb3954af99611b2bda8871fe8" candidate_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c711e5e8faf44e8bc7a9138e155773c" is_pre_election: true
I20260811 17:19:49.520212 22877 raft_consensus.cc:2454] T 9de18c9eb3954af99611b2bda8871fe8 P 7c711e5e8faf44e8bc7a9138e155773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1096fd23ccb24c7183fc39f31a1e00fc in term 0.
I20260811 17:19:49.520220 22656 raft_consensus.cc:2454] T 9de18c9eb3954af99611b2bda8871fe8 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1096fd23ccb24c7183fc39f31a1e00fc in term 0.
I20260811 17:19:49.520749 22353 leader_election.cc:302] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1096fd23ccb24c7183fc39f31a1e00fc, 7c711e5e8faf44e8bc7a9138e155773c, 9fbf9ebada614af9b5679b3e602cecd8; no voters:
I20260811 17:19:49.521104 23004 raft_consensus.cc:2781] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:19:49.521198 23004 raft_consensus.cc:482] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:19:49.521252 23004 raft_consensus.cc:3037] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:49.522337 23004 raft_consensus.cc:504] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } } peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } }
I20260811 17:19:49.522692 23004 leader_election.cc:288] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc [CANDIDATE]: Term 1 election: Requested vote from peers 9fbf9ebada614af9b5679b3e602cecd8 (127.9.157.130:40473), 7c711e5e8faf44e8bc7a9138e155773c (127.9.157.131:46347)
I20260811 17:19:49.523098 22877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9de18c9eb3954af99611b2bda8871fe8" candidate_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c711e5e8faf44e8bc7a9138e155773c"
I20260811 17:19:49.523240 22877 raft_consensus.cc:3037] T 9de18c9eb3954af99611b2bda8871fe8 P 7c711e5e8faf44e8bc7a9138e155773c [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:49.523789 22656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9de18c9eb3954af99611b2bda8871fe8" candidate_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fbf9ebada614af9b5679b3e602cecd8"
I20260811 17:19:49.523934 22656 raft_consensus.cc:3037] T 9de18c9eb3954af99611b2bda8871fe8 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:49.524510 22877 raft_consensus.cc:2454] T 9de18c9eb3954af99611b2bda8871fe8 P 7c711e5e8faf44e8bc7a9138e155773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1096fd23ccb24c7183fc39f31a1e00fc in term 1.
I20260811 17:19:49.524860 22353 leader_election.cc:302] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1096fd23ccb24c7183fc39f31a1e00fc, 7c711e5e8faf44e8bc7a9138e155773c; no voters:
I20260811 17:19:49.525058 23004 raft_consensus.cc:2781] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:49.525348 23004 raft_consensus.cc:686] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc [term 1 LEADER]: Becoming Leader. State: Replica: 1096fd23ccb24c7183fc39f31a1e00fc, State: Running, Role: LEADER
I20260811 17:19:49.525743 22656 raft_consensus.cc:2454] T 9de18c9eb3954af99611b2bda8871fe8 P 9fbf9ebada614af9b5679b3e602cecd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1096fd23ccb24c7183fc39f31a1e00fc in term 1.
I20260811 17:19:49.525746 23004 consensus_queue.cc:237] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } } peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } }
I20260811 17:19:49.529342 22201 catalog_manager.cc:5495] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc reported cstate change: term changed from 0 to 1, leader changed from <none> to 1096fd23ccb24c7183fc39f31a1e00fc (127.9.157.129). New cstate: current_term: 1 leader_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } health_report { overall_health: HEALTHY } } }
I20260811 17:19:49.547799 22989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.543404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48268 (local address 127.9.157.129:45711)
0811 17:19:49.543559 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:49.543565 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:49.543582 (+ 17us) server_negotiation.cc:407] Connection header received
0811 17:19:49.543698 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:49.543701 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:49.543769 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:49.543885 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:19:49.544246 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.545060 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.545916 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.546276 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.546377 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.546826 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:19:49.547340 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:19:49.547435 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:49.547535 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 17:19:49.547590 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 17:19:49.554080 22656 raft_consensus.cc:1261] T 9de18c9eb3954af99611b2bda8871fe8 P 9fbf9ebada614af9b5679b3e602cecd8 [term 1 FOLLOWER]: Refusing update from remote peer 1096fd23ccb24c7183fc39f31a1e00fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:49.554098 22877 raft_consensus.cc:1261] T 9de18c9eb3954af99611b2bda8871fe8 P 7c711e5e8faf44e8bc7a9138e155773c [term 1 FOLLOWER]: Refusing update from remote peer 1096fd23ccb24c7183fc39f31a1e00fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:49.554899 23014 consensus_queue.cc:1035] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:49.555176 23004 consensus_queue.cc:1035] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:49.564771 23021 mvcc.cc:205] Tried to move back new op lower bound from 7317376162005430272 to 7317376161900654592. Current Snapshot: MvccSnapshot[applied={T|T < 7317376162005430272}]
I20260811 17:19:49.567278 23019 mvcc.cc:205] Tried to move back new op lower bound from 7317376162005430272 to 7317376161900654592. Current Snapshot: MvccSnapshot[applied={T|T < 7317376162005430272 or (T in {7317376162005430272})}]
I20260811 17:19:49.573930 22201 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:19:49.574100 22201 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:19:49.577018 22201 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a15553ed49fb4a3d8511c2caf09b8aa6: Sending DeleteTablet for 3 replicas of tablet 9de18c9eb3954af99611b2bda8871fe8
I20260811 17:19:49.577960 22652 tablet_service.cc:1514] Processing DeleteTablet for tablet 9de18c9eb3954af99611b2bda8871fe8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:49 UTC) from {username='oryx', principal='oryx/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:56196
I20260811 17:19:49.577952 22851 tablet_service.cc:1514] Processing DeleteTablet for tablet 9de18c9eb3954af99611b2bda8871fe8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:49 UTC) from {username='oryx', principal='oryx/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:34336
I20260811 17:19:49.578066 22434 tablet_service.cc:1514] Processing DeleteTablet for tablet 9de18c9eb3954af99611b2bda8871fe8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:49 UTC) from {username='oryx', principal='oryx/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:48252
I20260811 17:19:49.578549 23033 tablet_replica.cc:330] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc: stopping tablet replica
I20260811 17:19:49.578547 23032 tablet_replica.cc:330] T 9de18c9eb3954af99611b2bda8871fe8 P 7c711e5e8faf44e8bc7a9138e155773c: stopping tablet replica
I20260811 17:19:49.578888 23032 raft_consensus.cc:2229] T 9de18c9eb3954af99611b2bda8871fe8 P 7c711e5e8faf44e8bc7a9138e155773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:19:49.578888 23033 raft_consensus.cc:2229] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:19:49.579249 23032 raft_consensus.cc:2258] T 9de18c9eb3954af99611b2bda8871fe8 P 7c711e5e8faf44e8bc7a9138e155773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:49.579259 23033 raft_consensus.cc:2258] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:49.579821 22201 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33014:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:19:49.580322 22201 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:19:49.580850 23032 ts_tablet_manager.cc:1905] T 9de18c9eb3954af99611b2bda8871fe8 P 7c711e5e8faf44e8bc7a9138e155773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:49.580850 23033 ts_tablet_manager.cc:1905] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:49.583770 23033 ts_tablet_manager.cc:1918] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:19:49.583885 23033 log.cc:1196] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-0/wal/wals/9de18c9eb3954af99611b2bda8871fe8
I20260811 17:19:49.584236 23033 ts_tablet_manager.cc:1939] T 9de18c9eb3954af99611b2bda8871fe8 P 1096fd23ccb24c7183fc39f31a1e00fc: Deleting consensus metadata
I20260811 17:19:49.584889 23035 tablet_replica.cc:330] T 9de18c9eb3954af99611b2bda8871fe8 P 9fbf9ebada614af9b5679b3e602cecd8: stopping tablet replica
I20260811 17:19:49.585350 23035 raft_consensus.cc:2229] T 9de18c9eb3954af99611b2bda8871fe8 P 9fbf9ebada614af9b5679b3e602cecd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:19:49.585569 22177 catalog_manager.cc:4841] TS 1096fd23ccb24c7183fc39f31a1e00fc (127.9.157.129:45711): tablet 9de18c9eb3954af99611b2bda8871fe8 (table test-table [id=849db498d63e44c8afba49acb5ee8482]) successfully deleted
I20260811 17:19:49.585822 23035 raft_consensus.cc:2258] T 9de18c9eb3954af99611b2bda8871fe8 P 9fbf9ebada614af9b5679b3e602cecd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:49.586347 22434 tablet_service.cc:1467] Processing CreateTablet for tablet 11af27a693c04a51bd4425bb549b2691 (DEFAULT_TABLE table=test-table [id=60bc952bb0bb4d02829b5bafeddb2593]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:49.586658 22434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11af27a693c04a51bd4425bb549b2691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:49.586882 22652 tablet_service.cc:1467] Processing CreateTablet for tablet 11af27a693c04a51bd4425bb549b2691 (DEFAULT_TABLE table=test-table [id=60bc952bb0bb4d02829b5bafeddb2593]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:49.587188 22652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11af27a693c04a51bd4425bb549b2691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:49.589170 22991 tablet_bootstrap.cc:492] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc: Bootstrap starting.
I20260811 17:19:49.589797 22995 tablet_bootstrap.cc:492] T 11af27a693c04a51bd4425bb549b2691 P 9fbf9ebada614af9b5679b3e602cecd8: Bootstrap starting.
I20260811 17:19:49.590075 22851 tablet_service.cc:1467] Processing CreateTablet for tablet 11af27a693c04a51bd4425bb549b2691 (DEFAULT_TABLE table=test-table [id=60bc952bb0bb4d02829b5bafeddb2593]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:49.590359 22991 tablet_bootstrap.cc:654] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:49.590436 22851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11af27a693c04a51bd4425bb549b2691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:49.590075 23032 ts_tablet_manager.cc:1918] T 9de18c9eb3954af99611b2bda8871fe8 P 7c711e5e8faf44e8bc7a9138e155773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:19:49.590523 23032 log.cc:1196] T 9de18c9eb3954af99611b2bda8871fe8 P 7c711e5e8faf44e8bc7a9138e155773c: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-2/wal/wals/9de18c9eb3954af99611b2bda8871fe8
I20260811 17:19:49.590885 23032 ts_tablet_manager.cc:1939] T 9de18c9eb3954af99611b2bda8871fe8 P 7c711e5e8faf44e8bc7a9138e155773c: Deleting consensus metadata
I20260811 17:19:49.590920 22995 tablet_bootstrap.cc:654] T 11af27a693c04a51bd4425bb549b2691 P 9fbf9ebada614af9b5679b3e602cecd8: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:49.592445 22180 catalog_manager.cc:4841] TS 7c711e5e8faf44e8bc7a9138e155773c (127.9.157.131:46347): tablet 9de18c9eb3954af99611b2bda8871fe8 (table test-table [id=849db498d63e44c8afba49acb5ee8482]) successfully deleted
I20260811 17:19:49.592953 22993 tablet_bootstrap.cc:492] T 11af27a693c04a51bd4425bb549b2691 P 7c711e5e8faf44e8bc7a9138e155773c: Bootstrap starting.
I20260811 17:19:49.593097 23035 ts_tablet_manager.cc:1905] T 9de18c9eb3954af99611b2bda8871fe8 P 9fbf9ebada614af9b5679b3e602cecd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:49.593971 22993 tablet_bootstrap.cc:654] T 11af27a693c04a51bd4425bb549b2691 P 7c711e5e8faf44e8bc7a9138e155773c: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:49.595336 22993 tablet_bootstrap.cc:492] T 11af27a693c04a51bd4425bb549b2691 P 7c711e5e8faf44e8bc7a9138e155773c: No bootstrap required, opened a new log
I20260811 17:19:49.595422 22993 ts_tablet_manager.cc:1397] T 11af27a693c04a51bd4425bb549b2691 P 7c711e5e8faf44e8bc7a9138e155773c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 17:19:49.595956 23035 ts_tablet_manager.cc:1918] T 9de18c9eb3954af99611b2bda8871fe8 P 9fbf9ebada614af9b5679b3e602cecd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:19:49.596066 23035 log.cc:1196] T 9de18c9eb3954af99611b2bda8871fe8 P 9fbf9ebada614af9b5679b3e602cecd8: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-1/wal/wals/9de18c9eb3954af99611b2bda8871fe8
I20260811 17:19:49.595984 22993 raft_consensus.cc:348] T 11af27a693c04a51bd4425bb549b2691 P 7c711e5e8faf44e8bc7a9138e155773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:49.596102 22993 raft_consensus.cc:374] T 11af27a693c04a51bd4425bb549b2691 P 7c711e5e8faf44e8bc7a9138e155773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:49.596144 22993 raft_consensus.cc:729] T 11af27a693c04a51bd4425bb549b2691 P 7c711e5e8faf44e8bc7a9138e155773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c711e5e8faf44e8bc7a9138e155773c, State: Initialized, Role: FOLLOWER
I20260811 17:19:49.596410 23035 ts_tablet_manager.cc:1939] T 9de18c9eb3954af99611b2bda8871fe8 P 9fbf9ebada614af9b5679b3e602cecd8: Deleting consensus metadata
I20260811 17:19:49.596367 22993 consensus_queue.cc:260] T 11af27a693c04a51bd4425bb549b2691 P 7c711e5e8faf44e8bc7a9138e155773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:49.597357 22993 ts_tablet_manager.cc:1428] T 11af27a693c04a51bd4425bb549b2691 P 7c711e5e8faf44e8bc7a9138e155773c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 17:19:49.597800 22183 catalog_manager.cc:4841] TS 9fbf9ebada614af9b5679b3e602cecd8 (127.9.157.130:40473): tablet 9de18c9eb3954af99611b2bda8871fe8 (table test-table [id=849db498d63e44c8afba49acb5ee8482]) successfully deleted
I20260811 17:19:49.598508 22995 tablet_bootstrap.cc:492] T 11af27a693c04a51bd4425bb549b2691 P 9fbf9ebada614af9b5679b3e602cecd8: No bootstrap required, opened a new log
I20260811 17:19:49.598603 22995 ts_tablet_manager.cc:1397] T 11af27a693c04a51bd4425bb549b2691 P 9fbf9ebada614af9b5679b3e602cecd8: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260811 17:19:49.598948 22991 tablet_bootstrap.cc:492] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc: No bootstrap required, opened a new log
I20260811 17:19:49.599043 22991 ts_tablet_manager.cc:1397] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.001s
I20260811 17:19:49.599133 22995 raft_consensus.cc:348] T 11af27a693c04a51bd4425bb549b2691 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:49.599273 22995 raft_consensus.cc:374] T 11af27a693c04a51bd4425bb549b2691 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:49.599310 22995 raft_consensus.cc:729] T 11af27a693c04a51bd4425bb549b2691 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fbf9ebada614af9b5679b3e602cecd8, State: Initialized, Role: FOLLOWER
I20260811 17:19:49.599480 22995 consensus_queue.cc:260] T 11af27a693c04a51bd4425bb549b2691 P 9fbf9ebada614af9b5679b3e602cecd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:49.599576 22991 raft_consensus.cc:348] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:49.599725 22991 raft_consensus.cc:374] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:49.599767 22991 raft_consensus.cc:729] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1096fd23ccb24c7183fc39f31a1e00fc, State: Initialized, Role: FOLLOWER
I20260811 17:19:49.599828 22995 ts_tablet_manager.cc:1428] T 11af27a693c04a51bd4425bb549b2691 P 9fbf9ebada614af9b5679b3e602cecd8: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 17:19:49.599929 22991 consensus_queue.cc:260] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:49.600240 22991 ts_tablet_manager.cc:1428] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
W20260811 17:19:49.636686 22549 tablet.cc:2369] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:19:49.636793 22952 tablet.cc:2369] T 11af27a693c04a51bd4425bb549b2691 P 7c711e5e8faf44e8bc7a9138e155773c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:19:49.639225 22746 tablet.cc:2369] T 11af27a693c04a51bd4425bb549b2691 P 9fbf9ebada614af9b5679b3e602cecd8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:19:49.735064 23014 raft_consensus.cc:482] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:19:49.735250 23014 raft_consensus.cc:504] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:49.735770 23014 leader_election.cc:288] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9fbf9ebada614af9b5679b3e602cecd8 (127.9.157.130:40473), 7c711e5e8faf44e8bc7a9138e155773c (127.9.157.131:46347)
I20260811 17:19:49.736383 22656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11af27a693c04a51bd4425bb549b2691" candidate_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fbf9ebada614af9b5679b3e602cecd8" is_pre_election: true
I20260811 17:19:49.736384 22877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11af27a693c04a51bd4425bb549b2691" candidate_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c711e5e8faf44e8bc7a9138e155773c" is_pre_election: true
I20260811 17:19:49.736550 22656 raft_consensus.cc:2454] T 11af27a693c04a51bd4425bb549b2691 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1096fd23ccb24c7183fc39f31a1e00fc in term 0.
I20260811 17:19:49.736580 22877 raft_consensus.cc:2454] T 11af27a693c04a51bd4425bb549b2691 P 7c711e5e8faf44e8bc7a9138e155773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1096fd23ccb24c7183fc39f31a1e00fc in term 0.
I20260811 17:19:49.736879 22354 leader_election.cc:302] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1096fd23ccb24c7183fc39f31a1e00fc, 9fbf9ebada614af9b5679b3e602cecd8; no voters:
I20260811 17:19:49.737102 23014 raft_consensus.cc:2781] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:19:49.737174 23014 raft_consensus.cc:482] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:19:49.737221 23014 raft_consensus.cc:3037] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:49.738498 23014 raft_consensus.cc:504] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:49.738915 23014 leader_election.cc:288] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc [CANDIDATE]: Term 1 election: Requested vote from peers 9fbf9ebada614af9b5679b3e602cecd8 (127.9.157.130:40473), 7c711e5e8faf44e8bc7a9138e155773c (127.9.157.131:46347)
I20260811 17:19:49.739370 22877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11af27a693c04a51bd4425bb549b2691" candidate_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c711e5e8faf44e8bc7a9138e155773c"
I20260811 17:19:49.739418 22656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11af27a693c04a51bd4425bb549b2691" candidate_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fbf9ebada614af9b5679b3e602cecd8"
I20260811 17:19:49.739526 22877 raft_consensus.cc:3037] T 11af27a693c04a51bd4425bb549b2691 P 7c711e5e8faf44e8bc7a9138e155773c [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:49.739539 22656 raft_consensus.cc:3037] T 11af27a693c04a51bd4425bb549b2691 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:49.740615 22877 raft_consensus.cc:2454] T 11af27a693c04a51bd4425bb549b2691 P 7c711e5e8faf44e8bc7a9138e155773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1096fd23ccb24c7183fc39f31a1e00fc in term 1.
I20260811 17:19:49.740721 22656 raft_consensus.cc:2454] T 11af27a693c04a51bd4425bb549b2691 P 9fbf9ebada614af9b5679b3e602cecd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1096fd23ccb24c7183fc39f31a1e00fc in term 1.
I20260811 17:19:49.741008 22353 leader_election.cc:302] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1096fd23ccb24c7183fc39f31a1e00fc, 7c711e5e8faf44e8bc7a9138e155773c; no voters:
I20260811 17:19:49.741281 23014 raft_consensus.cc:2781] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:49.741408 23014 raft_consensus.cc:686] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc [term 1 LEADER]: Becoming Leader. State: Replica: 1096fd23ccb24c7183fc39f31a1e00fc, State: Running, Role: LEADER
I20260811 17:19:49.741554 23014 consensus_queue.cc:237] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:49.743518 22203 catalog_manager.cc:5495] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc reported cstate change: term changed from 0 to 1, leader changed from <none> to 1096fd23ccb24c7183fc39f31a1e00fc (127.9.157.129). New cstate: current_term: 1 leader_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } health_report { overall_health: UNKNOWN } } }
Aug 11 17:19:49 dist-test-slave-mjjr krb5kdc[22128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468788, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.190@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM
I20260811 17:19:49.818730 23046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.809660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44807 (local address 127.0.0.1:33024)
0811 17:19:49.810023 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:49.810037 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:49.810149 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:49.810423 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:49.810427 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:49.810856 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:49.810962 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:49.810968 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.811748 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.811750 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:49.812312 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:49.812320 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.812727 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.812730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:49.812797 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.813336 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:49.813348 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:49.814969 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:49.816916 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:49.816934 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:49.816944 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:49.817213 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:49.817511 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:49.817516 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:49.817518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:49.817603 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:49.818025 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:49.818036 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:49.818160 (+ 124us) client_negotiation.cc:769] Sending connection context
0811 17:19:49.818283 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 17:19:49.818586 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":121,"thread_start_us":119,"threads_started":1}
I20260811 17:19:49.818728 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.809832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33024 (local address 127.9.157.190:44807)
0811 17:19:49.810016 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:49.810023 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:49.810118 (+ 95us) server_negotiation.cc:407] Connection header received
0811 17:19:49.810235 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:49.810238 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:49.810279 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:49.810351 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:49.811078 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.811588 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.812413 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.812621 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.812697 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.815150 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:49.815153 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:49.815155 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:49.815177 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:49.816775 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:49.817320 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:49.817323 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:49.817326 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:49.817418 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:49.817701 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:49.817704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:49.817706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:49.817912 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:49.818023 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:49.818331 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 17:19:49.818524 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":97}
I20260811 17:19:49.823236 22203 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:33024:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 17:19:49.824047 22203 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:19:49.829015 22851 tablet_service.cc:1467] Processing CreateTablet for tablet 29ca176313414efab91bd69e9e553606 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=16c12d15acc94e899c1ad82b3d69c3dd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 17:19:49.829041 22652 tablet_service.cc:1467] Processing CreateTablet for tablet 29ca176313414efab91bd69e9e553606 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=16c12d15acc94e899c1ad82b3d69c3dd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 17:19:49.829481 22851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29ca176313414efab91bd69e9e553606. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:49.829481 22652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29ca176313414efab91bd69e9e553606. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:49.831887 22434 tablet_service.cc:1467] Processing CreateTablet for tablet 29ca176313414efab91bd69e9e553606 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=16c12d15acc94e899c1ad82b3d69c3dd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 17:19:49.832284 22434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29ca176313414efab91bd69e9e553606. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:49.832722 22995 tablet_bootstrap.cc:492] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8: Bootstrap starting.
I20260811 17:19:49.833739 22995 tablet_bootstrap.cc:654] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:49.834451 22993 tablet_bootstrap.cc:492] T 29ca176313414efab91bd69e9e553606 P 7c711e5e8faf44e8bc7a9138e155773c: Bootstrap starting.
I20260811 17:19:49.835460 22991 tablet_bootstrap.cc:492] T 29ca176313414efab91bd69e9e553606 P 1096fd23ccb24c7183fc39f31a1e00fc: Bootstrap starting.
I20260811 17:19:49.835528 22993 tablet_bootstrap.cc:654] T 29ca176313414efab91bd69e9e553606 P 7c711e5e8faf44e8bc7a9138e155773c: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:49.836422 22991 tablet_bootstrap.cc:654] T 29ca176313414efab91bd69e9e553606 P 1096fd23ccb24c7183fc39f31a1e00fc: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:49.836454 22995 tablet_bootstrap.cc:492] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8: No bootstrap required, opened a new log
I20260811 17:19:49.836567 22995 ts_tablet_manager.cc:1397] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 17:19:49.837072 22993 tablet_bootstrap.cc:492] T 29ca176313414efab91bd69e9e553606 P 7c711e5e8faf44e8bc7a9138e155773c: No bootstrap required, opened a new log
I20260811 17:19:49.837177 22993 ts_tablet_manager.cc:1397] T 29ca176313414efab91bd69e9e553606 P 7c711e5e8faf44e8bc7a9138e155773c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 17:19:49.837116 22995 raft_consensus.cc:348] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:49.837239 22995 raft_consensus.cc:374] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:49.837291 22995 raft_consensus.cc:729] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fbf9ebada614af9b5679b3e602cecd8, State: Initialized, Role: FOLLOWER
I20260811 17:19:49.837441 22995 consensus_queue.cc:260] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:49.837668 22993 raft_consensus.cc:348] T 29ca176313414efab91bd69e9e553606 P 7c711e5e8faf44e8bc7a9138e155773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:49.837779 22991 tablet_bootstrap.cc:492] T 29ca176313414efab91bd69e9e553606 P 1096fd23ccb24c7183fc39f31a1e00fc: No bootstrap required, opened a new log
I20260811 17:19:49.837797 22993 raft_consensus.cc:374] T 29ca176313414efab91bd69e9e553606 P 7c711e5e8faf44e8bc7a9138e155773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:49.837726 23005 tablet_replica.cc:439] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } } }
I20260811 17:19:49.837832 22993 raft_consensus.cc:729] T 29ca176313414efab91bd69e9e553606 P 7c711e5e8faf44e8bc7a9138e155773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c711e5e8faf44e8bc7a9138e155773c, State: Initialized, Role: FOLLOWER
I20260811 17:19:49.837882 22991 ts_tablet_manager.cc:1397] T 29ca176313414efab91bd69e9e553606 P 1096fd23ccb24c7183fc39f31a1e00fc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 17:19:49.837875 23005 tablet_replica.cc:442] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:19:49.838002 22993 consensus_queue.cc:260] T 29ca176313414efab91bd69e9e553606 P 7c711e5e8faf44e8bc7a9138e155773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:49.838138 22995 ts_tablet_manager.cc:1428] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 17:19:49.838243 23003 tablet_replica.cc:439] T 29ca176313414efab91bd69e9e553606 P 7c711e5e8faf44e8bc7a9138e155773c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } } }
I20260811 17:19:49.838322 22991 raft_consensus.cc:348] T 29ca176313414efab91bd69e9e553606 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:49.838442 22991 raft_consensus.cc:374] T 29ca176313414efab91bd69e9e553606 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:49.838447 23003 tablet_replica.cc:442] T 29ca176313414efab91bd69e9e553606 P 7c711e5e8faf44e8bc7a9138e155773c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:19:49.838482 22991 raft_consensus.cc:729] T 29ca176313414efab91bd69e9e553606 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1096fd23ccb24c7183fc39f31a1e00fc, State: Initialized, Role: FOLLOWER
I20260811 17:19:49.838617 22991 consensus_queue.cc:260] T 29ca176313414efab91bd69e9e553606 P 1096fd23ccb24c7183fc39f31a1e00fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:49.838699 22993 ts_tablet_manager.cc:1428] T 29ca176313414efab91bd69e9e553606 P 7c711e5e8faf44e8bc7a9138e155773c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 17:19:49.839797 23014 tablet_replica.cc:439] T 29ca176313414efab91bd69e9e553606 P 1096fd23ccb24c7183fc39f31a1e00fc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } } }
I20260811 17:19:49.839941 23014 tablet_replica.cc:442] T 29ca176313414efab91bd69e9e553606 P 1096fd23ccb24c7183fc39f31a1e00fc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:19:49.840279 22991 ts_tablet_manager.cc:1428] T 29ca176313414efab91bd69e9e553606 P 1096fd23ccb24c7183fc39f31a1e00fc: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 17:19:49.922050 23005 raft_consensus.cc:482] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:19:49.922212 23005 raft_consensus.cc:504] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:49.923359 23005 leader_election.cc:288] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1096fd23ccb24c7183fc39f31a1e00fc (127.9.157.129:45711), 7c711e5e8faf44e8bc7a9138e155773c (127.9.157.131:46347)
I20260811 17:19:49.928129 23052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.923763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:45711 (local address 127.9.157.130:53819)
0811 17:19:49.924177 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:49.924189 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:49.924272 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:49.924573 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:49.924576 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:49.924592 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:49.924775 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:49.924782 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.925602 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.925605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:49.926584 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:49.926595 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.927514 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.927518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:49.927695 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.927731 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 17:19:49.927882 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 17:19:49.927978 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":59,"thread_start_us":125,"threads_started":1}
I20260811 17:19:49.928182 22989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.924040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:53819 (local address 127.9.157.129:45711)
0811 17:19:49.924206 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:49.924212 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:49.924233 (+ 21us) server_negotiation.cc:407] Connection header received
0811 17:19:49.924398 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:49.924403 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:49.924463 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:49.924530 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:49.924900 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.925454 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.926747 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.927211 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.927328 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.927381 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:49.927907 (+ 526us) server_negotiation.cc:299] Negotiation successful
0811 17:19:49.928011 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 17:19:49.928157 22990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.923823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:58711 (local address 127.9.157.131:46347)
0811 17:19:49.923984 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:49.923989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:49.924147 (+ 158us) server_negotiation.cc:407] Connection header received
0811 17:19:49.924439 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:49.924442 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:49.924483 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:49.924583 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:19:49.924953 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.925451 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.926787 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.927223 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.927558 (+ 335us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.927624 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:49.927945 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 17:19:49.928028 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 17:19:49.928633 23051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.923685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:46347 (local address 127.9.157.130:58711)
0811 17:19:49.924004 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:49.924023 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:49.924269 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:49.924617 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:49.924620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:49.924639 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:19:49.924807 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:49.924813 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.925572 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.925579 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:49.926620 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:49.926632 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.927520 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.927524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:49.927715 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.927752 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 17:19:49.928338 (+ 586us) client_negotiation.cc:240] Negotiation successful
0811 17:19:49.928440 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":85,"thread_start_us":115,"threads_started":1}
I20260811 17:19:49.929003 22470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29ca176313414efab91bd69e9e553606" candidate_uuid: "9fbf9ebada614af9b5679b3e602cecd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" is_pre_election: true
I20260811 17:19:49.929171 22877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29ca176313414efab91bd69e9e553606" candidate_uuid: "9fbf9ebada614af9b5679b3e602cecd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c711e5e8faf44e8bc7a9138e155773c" is_pre_election: true
I20260811 17:19:49.929303 22470 raft_consensus.cc:2454] T 29ca176313414efab91bd69e9e553606 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fbf9ebada614af9b5679b3e602cecd8 in term 0.
I20260811 17:19:49.929371 22877 raft_consensus.cc:2454] T 29ca176313414efab91bd69e9e553606 P 7c711e5e8faf44e8bc7a9138e155773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fbf9ebada614af9b5679b3e602cecd8 in term 0.
I20260811 17:19:49.929800 22583 leader_election.cc:302] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1096fd23ccb24c7183fc39f31a1e00fc, 7c711e5e8faf44e8bc7a9138e155773c, 9fbf9ebada614af9b5679b3e602cecd8; no voters:
I20260811 17:19:49.930079 23005 raft_consensus.cc:2781] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:19:49.930145 23005 raft_consensus.cc:482] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:19:49.930182 23005 raft_consensus.cc:3037] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:49.931301 23005 raft_consensus.cc:504] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:49.931795 23005 leader_election.cc:288] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8 [CANDIDATE]: Term 1 election: Requested vote from peers 1096fd23ccb24c7183fc39f31a1e00fc (127.9.157.129:45711), 7c711e5e8faf44e8bc7a9138e155773c (127.9.157.131:46347)
I20260811 17:19:49.932250 22877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29ca176313414efab91bd69e9e553606" candidate_uuid: "9fbf9ebada614af9b5679b3e602cecd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c711e5e8faf44e8bc7a9138e155773c"
I20260811 17:19:49.932279 22470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29ca176313414efab91bd69e9e553606" candidate_uuid: "9fbf9ebada614af9b5679b3e602cecd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1096fd23ccb24c7183fc39f31a1e00fc"
I20260811 17:19:49.932407 22877 raft_consensus.cc:3037] T 29ca176313414efab91bd69e9e553606 P 7c711e5e8faf44e8bc7a9138e155773c [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:49.932430 22470 raft_consensus.cc:3037] T 29ca176313414efab91bd69e9e553606 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:49.933434 22877 raft_consensus.cc:2454] T 29ca176313414efab91bd69e9e553606 P 7c711e5e8faf44e8bc7a9138e155773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fbf9ebada614af9b5679b3e602cecd8 in term 1.
I20260811 17:19:49.933475 22470 raft_consensus.cc:2454] T 29ca176313414efab91bd69e9e553606 P 1096fd23ccb24c7183fc39f31a1e00fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fbf9ebada614af9b5679b3e602cecd8 in term 1.
I20260811 17:19:49.933835 22586 leader_election.cc:302] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c711e5e8faf44e8bc7a9138e155773c, 9fbf9ebada614af9b5679b3e602cecd8; no voters:
I20260811 17:19:49.934032 23005 raft_consensus.cc:2781] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:49.934229 23005 raft_consensus.cc:686] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8 [term 1 LEADER]: Becoming Leader. State: Replica: 9fbf9ebada614af9b5679b3e602cecd8, State: Running, Role: LEADER
I20260811 17:19:49.934517 23005 consensus_queue.cc:237] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:49.935848 23053 tablet_replica.cc:439] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8: TxnStatusTablet state changed. Reason: New leader 9fbf9ebada614af9b5679b3e602cecd8. Latest consensus state: current_term: 1 leader_uuid: "9fbf9ebada614af9b5679b3e602cecd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } } }
I20260811 17:19:49.935967 23053 tablet_replica.cc:442] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8: This TxnStatusTablet replica's current role is: LEADER
I20260811 17:19:49.936285 23055 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 17:19:49.936458 23055 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 17:19:49.937294 22201 catalog_manager.cc:5495] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9fbf9ebada614af9b5679b3e602cecd8 (127.9.157.130). New cstate: current_term: 1 leader_uuid: "9fbf9ebada614af9b5679b3e602cecd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } health_report { overall_health: UNKNOWN } } }
I20260811 17:19:49.968133 23046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.963889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:40473 (local address 127.0.0.1:56198)
0811 17:19:49.964043 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:49.964060 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:49.964168 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:49.964529 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:49.964531 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:49.964537 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 17:19:49.964631 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:49.964636 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.965660 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.965662 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:49.966280 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:49.966288 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.966859 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.966861 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:49.966941 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.966992 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 17:19:49.967845 (+ 853us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 17:19:49.967858 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 17:19:49.967928 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 17:19:49.968010 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 17:19:49.968153 22986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.964004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56198 (local address 127.9.157.130:40473)
0811 17:19:49.964144 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:49.964149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:49.964166 (+ 17us) server_negotiation.cc:407] Connection header received
0811 17:19:49.964326 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:49.964330 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:49.964400 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:49.964512 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:19:49.964746 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.965555 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.966403 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.966746 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.966856 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.967147 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:19:49.967752 (+ 605us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:19:49.967874 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:49.967947 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 17:19:49.968009 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 17:19:49.971534 22877 raft_consensus.cc:1261] T 29ca176313414efab91bd69e9e553606 P 7c711e5e8faf44e8bc7a9138e155773c [term 1 FOLLOWER]: Refusing update from remote peer 9fbf9ebada614af9b5679b3e602cecd8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:49.971696 22470 raft_consensus.cc:1261] T 29ca176313414efab91bd69e9e553606 P 1096fd23ccb24c7183fc39f31a1e00fc [term 1 FOLLOWER]: Refusing update from remote peer 9fbf9ebada614af9b5679b3e602cecd8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:49.972110 23053 consensus_queue.cc:1035] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:49.972277 23005 consensus_queue.cc:1035] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:49.974155 23014 tablet_replica.cc:439] T 29ca176313414efab91bd69e9e553606 P 1096fd23ccb24c7183fc39f31a1e00fc: TxnStatusTablet state changed. Reason: New leader 9fbf9ebada614af9b5679b3e602cecd8. Latest consensus state: current_term: 1 leader_uuid: "9fbf9ebada614af9b5679b3e602cecd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } } }
I20260811 17:19:49.974293 23014 tablet_replica.cc:442] T 29ca176313414efab91bd69e9e553606 P 1096fd23ccb24c7183fc39f31a1e00fc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:19:49.974695 23003 tablet_replica.cc:439] T 29ca176313414efab91bd69e9e553606 P 7c711e5e8faf44e8bc7a9138e155773c: TxnStatusTablet state changed. Reason: New leader 9fbf9ebada614af9b5679b3e602cecd8. Latest consensus state: current_term: 1 leader_uuid: "9fbf9ebada614af9b5679b3e602cecd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } } }
I20260811 17:19:49.974829 23003 tablet_replica.cc:442] T 29ca176313414efab91bd69e9e553606 P 7c711e5e8faf44e8bc7a9138e155773c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:19:49.977210 23005 tablet_replica.cc:439] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9fbf9ebada614af9b5679b3e602cecd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } } }
I20260811 17:19:49.977339 23005 tablet_replica.cc:442] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8: This TxnStatusTablet replica's current role is: LEADER
I20260811 17:19:49.977465 23005 tablet_replica.cc:439] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9fbf9ebada614af9b5679b3e602cecd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } } }
I20260811 17:19:49.977550 23005 tablet_replica.cc:442] T 29ca176313414efab91bd69e9e553606 P 9fbf9ebada614af9b5679b3e602cecd8: This TxnStatusTablet replica's current role is: LEADER
I20260811 17:19:49.977653 23014 tablet_replica.cc:439] T 29ca176313414efab91bd69e9e553606 P 1096fd23ccb24c7183fc39f31a1e00fc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9fbf9ebada614af9b5679b3e602cecd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } } }
I20260811 17:19:49.977766 23014 tablet_replica.cc:442] T 29ca176313414efab91bd69e9e553606 P 1096fd23ccb24c7183fc39f31a1e00fc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:19:49.981235 23003 tablet_replica.cc:439] T 29ca176313414efab91bd69e9e553606 P 7c711e5e8faf44e8bc7a9138e155773c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9fbf9ebada614af9b5679b3e602cecd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } } }
I20260811 17:19:49.981364 23003 tablet_replica.cc:442] T 29ca176313414efab91bd69e9e553606 P 7c711e5e8faf44e8bc7a9138e155773c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 17:19:49.994159 23067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.990056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:40473 (local address 127.9.157.129:45885)
0811 17:19:49.990499 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:49.990517 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:49.990633 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:49.990903 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:49.990906 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:49.990912 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 17:19:49.991007 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:49.991011 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.992064 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.992067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:49.992844 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:49.992854 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.993452 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.993454 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:49.993516 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.993566 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 17:19:49.993887 (+ 321us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 17:19:49.993892 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 17:19:49.993945 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 17:19:49.994018 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":162,"threads_started":1}
I20260811 17:19:49.994166 22986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:49.990217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:45885 (local address 127.9.157.130:40473)
0811 17:19:49.990357 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:49.990362 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:49.990586 (+ 224us) server_negotiation.cc:407] Connection header received
0811 17:19:49.990739 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:49.990743 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:49.990802 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:49.990887 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:19:49.991116 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.991952 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.992964 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:49.993324 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:49.993432 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:49.993708 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:19:49.993799 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:19:49.993871 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:49.993963 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 17:19:49.994022 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 17:19:50.001581 22656 raft_consensus.cc:1261] T 11af27a693c04a51bd4425bb549b2691 P 9fbf9ebada614af9b5679b3e602cecd8 [term 1 FOLLOWER]: Refusing update from remote peer 1096fd23ccb24c7183fc39f31a1e00fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:50.001595 22877 raft_consensus.cc:1261] T 11af27a693c04a51bd4425bb549b2691 P 7c711e5e8faf44e8bc7a9138e155773c [term 1 FOLLOWER]: Refusing update from remote peer 1096fd23ccb24c7183fc39f31a1e00fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:50.002003 23024 consensus_queue.cc:1035] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:50.002187 23014 consensus_queue.cc:1035] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:50.008800 23018 mvcc.cc:205] Tried to move back new op lower bound from 7317376163842465792 to 7317376162783166464. Current Snapshot: MvccSnapshot[applied={T|T < 7317376163842465792}]
I20260811 17:19:50.027707 22989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:50.023615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:42905 (local address 127.9.157.129:45711)
0811 17:19:50.023786 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:50.023791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:50.024081 (+ 290us) server_negotiation.cc:407] Connection header received
0811 17:19:50.024210 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:50.024212 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:50.024288 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:50.024368 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:19:50.024639 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:50.025299 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:50.026477 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:50.026706 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:50.026823 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:50.027178 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:19:50.027283 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:19:50.027364 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:50.027462 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 17:19:50.027528 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 17:19:50.027725 23072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:50.023541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:45711 (local address 127.9.157.130:42905)
0811 17:19:50.023983 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:50.023999 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:50.024127 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:50.024395 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:50.024398 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:50.024406 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 17:19:50.024537 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:50.024544 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:50.025406 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:50.025412 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:50.026346 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:50.026357 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:50.026835 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:50.026838 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:50.026945 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:50.027004 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 17:19:50.027388 (+ 384us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 17:19:50.027395 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 17:19:50.027451 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 17:19:50.027523 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":193,"threads_started":1}
I20260811 17:19:50.053186 22201 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:19:50.053383 22201 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:19:50.055693 22201 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a15553ed49fb4a3d8511c2caf09b8aa6: Sending DeleteTablet for 3 replicas of tablet 11af27a693c04a51bd4425bb549b2691
I20260811 17:19:50.056291 22434 tablet_service.cc:1514] Processing DeleteTablet for tablet 11af27a693c04a51bd4425bb549b2691 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:50 UTC) from {username='oryx', principal='oryx/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:48252
I20260811 17:19:50.056511 23033 tablet_replica.cc:330] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc: stopping tablet replica
I20260811 17:19:50.056512 22652 tablet_service.cc:1514] Processing DeleteTablet for tablet 11af27a693c04a51bd4425bb549b2691 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:50 UTC) from {username='oryx', principal='oryx/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:56196
I20260811 17:19:50.056665 23033 raft_consensus.cc:2229] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:19:50.056665 22851 tablet_service.cc:1514] Processing DeleteTablet for tablet 11af27a693c04a51bd4425bb549b2691 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:50 UTC) from {username='oryx', principal='oryx/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:34336
I20260811 17:19:50.056890 23033 raft_consensus.cc:2258] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:50.057082 23035 tablet_replica.cc:330] T 11af27a693c04a51bd4425bb549b2691 P 9fbf9ebada614af9b5679b3e602cecd8: stopping tablet replica
I20260811 17:19:50.057190 23035 raft_consensus.cc:2229] T 11af27a693c04a51bd4425bb549b2691 P 9fbf9ebada614af9b5679b3e602cecd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:19:50.057282 23035 raft_consensus.cc:2258] T 11af27a693c04a51bd4425bb549b2691 P 9fbf9ebada614af9b5679b3e602cecd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:50.057469 23032 tablet_replica.cc:330] T 11af27a693c04a51bd4425bb549b2691 P 7c711e5e8faf44e8bc7a9138e155773c: stopping tablet replica
I20260811 17:19:50.057591 23032 raft_consensus.cc:2229] T 11af27a693c04a51bd4425bb549b2691 P 7c711e5e8faf44e8bc7a9138e155773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:19:50.057729 23032 raft_consensus.cc:2258] T 11af27a693c04a51bd4425bb549b2691 P 7c711e5e8faf44e8bc7a9138e155773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:50.058151 23035 ts_tablet_manager.cc:1905] T 11af27a693c04a51bd4425bb549b2691 P 9fbf9ebada614af9b5679b3e602cecd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:50.058609 23033 ts_tablet_manager.cc:1905] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:50.059556 23032 ts_tablet_manager.cc:1905] T 11af27a693c04a51bd4425bb549b2691 P 7c711e5e8faf44e8bc7a9138e155773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:50.061811 23035 ts_tablet_manager.cc:1918] T 11af27a693c04a51bd4425bb549b2691 P 9fbf9ebada614af9b5679b3e602cecd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:19:50.061911 23035 log.cc:1196] T 11af27a693c04a51bd4425bb549b2691 P 9fbf9ebada614af9b5679b3e602cecd8: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-1/wal/wals/11af27a693c04a51bd4425bb549b2691
I20260811 17:19:50.061929 23033 ts_tablet_manager.cc:1918] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:19:50.062007 23033 log.cc:1196] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-0/wal/wals/11af27a693c04a51bd4425bb549b2691
I20260811 17:19:50.062224 23035 ts_tablet_manager.cc:1939] T 11af27a693c04a51bd4425bb549b2691 P 9fbf9ebada614af9b5679b3e602cecd8: Deleting consensus metadata
I20260811 17:19:50.062271 23033 ts_tablet_manager.cc:1939] T 11af27a693c04a51bd4425bb549b2691 P 1096fd23ccb24c7183fc39f31a1e00fc: Deleting consensus metadata
I20260811 17:19:50.062770 23032 ts_tablet_manager.cc:1918] T 11af27a693c04a51bd4425bb549b2691 P 7c711e5e8faf44e8bc7a9138e155773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:19:50.062845 23032 log.cc:1196] T 11af27a693c04a51bd4425bb549b2691 P 7c711e5e8faf44e8bc7a9138e155773c: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-2/wal/wals/11af27a693c04a51bd4425bb549b2691
I20260811 17:19:50.063114 23032 ts_tablet_manager.cc:1939] T 11af27a693c04a51bd4425bb549b2691 P 7c711e5e8faf44e8bc7a9138e155773c: Deleting consensus metadata
I20260811 17:19:50.063199 22183 catalog_manager.cc:4841] TS 9fbf9ebada614af9b5679b3e602cecd8 (127.9.157.130:40473): tablet 11af27a693c04a51bd4425bb549b2691 (table test-table [id=60bc952bb0bb4d02829b5bafeddb2593]) successfully deleted
I20260811 17:19:50.063210 22177 catalog_manager.cc:4841] TS 1096fd23ccb24c7183fc39f31a1e00fc (127.9.157.129:45711): tablet 11af27a693c04a51bd4425bb549b2691 (table test-table [id=60bc952bb0bb4d02829b5bafeddb2593]) successfully deleted
I20260811 17:19:50.063884 22180 catalog_manager.cc:4841] TS 7c711e5e8faf44e8bc7a9138e155773c (127.9.157.131:46347): tablet 11af27a693c04a51bd4425bb549b2691 (table test-table [id=60bc952bb0bb4d02829b5bafeddb2593]) successfully deleted
Aug 11 17:19:50 dist-test-slave-mjjr krb5kdc[22128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 17:19:50 dist-test-slave-mjjr krb5kdc[22128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:19:50 dist-test-slave-mjjr krb5kdc[22128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 17:19:50.092365 22288 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:50.085228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33028 (local address 127.9.157.190:44807)
0811 17:19:50.085401 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:50.085405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:50.085546 (+ 141us) server_negotiation.cc:407] Connection header received
0811 17:19:50.085662 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:50.085666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:50.085728 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:50.085832 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:50.086576 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:50.087405 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:50.088367 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:50.088695 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:50.088818 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:50.092204 (+ 3386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33028: BlockingRecv error: recv got EOF from 127.0.0.1:33028 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 11 17:19:50 dist-test-slave-mjjr krb5kdc[22128](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM
I20260811 17:19:50.103094 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:50.093989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33044 (local address 127.9.157.190:44807)
0811 17:19:50.094150 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:50.094156 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:50.094449 (+ 293us) server_negotiation.cc:407] Connection header received
0811 17:19:50.094555 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:50.094558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:50.094613 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:50.094701 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:50.095329 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:50.096170 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:50.096953 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:50.097302 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:50.097412 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:50.099890 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:50.099894 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:50.099897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:50.099930 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:50.101502 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:50.101906 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:50.101910 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:50.101911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:50.101998 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:50.102241 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:50.102244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:50.102246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:50.102469 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:50.102599 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:50.102800 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 17:19:50.102922 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 17:19:50.106079 22201 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33044:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:19:50.106642 22201 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:19:50.111919 22652 tablet_service.cc:1467] Processing CreateTablet for tablet 3b56c5a5163f4b809b304bbf09c5d596 (DEFAULT_TABLE table=test-table [id=5d20248272bb4c309bec8ca0ad64694c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:50.111948 22434 tablet_service.cc:1467] Processing CreateTablet for tablet 3b56c5a5163f4b809b304bbf09c5d596 (DEFAULT_TABLE table=test-table [id=5d20248272bb4c309bec8ca0ad64694c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:50.111989 22851 tablet_service.cc:1467] Processing CreateTablet for tablet 3b56c5a5163f4b809b304bbf09c5d596 (DEFAULT_TABLE table=test-table [id=5d20248272bb4c309bec8ca0ad64694c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:50.112262 22851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b56c5a5163f4b809b304bbf09c5d596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:50.112263 22434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b56c5a5163f4b809b304bbf09c5d596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:50.112262 22652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b56c5a5163f4b809b304bbf09c5d596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:50.114763 22993 tablet_bootstrap.cc:492] T 3b56c5a5163f4b809b304bbf09c5d596 P 7c711e5e8faf44e8bc7a9138e155773c: Bootstrap starting.
I20260811 17:19:50.114874 22995 tablet_bootstrap.cc:492] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8: Bootstrap starting.
I20260811 17:19:50.115796 22993 tablet_bootstrap.cc:654] T 3b56c5a5163f4b809b304bbf09c5d596 P 7c711e5e8faf44e8bc7a9138e155773c: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:50.115796 22995 tablet_bootstrap.cc:654] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:50.116662 22991 tablet_bootstrap.cc:492] T 3b56c5a5163f4b809b304bbf09c5d596 P 1096fd23ccb24c7183fc39f31a1e00fc: Bootstrap starting.
I20260811 17:19:50.117513 22991 tablet_bootstrap.cc:654] T 3b56c5a5163f4b809b304bbf09c5d596 P 1096fd23ccb24c7183fc39f31a1e00fc: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:50.119082 22993 tablet_bootstrap.cc:492] T 3b56c5a5163f4b809b304bbf09c5d596 P 7c711e5e8faf44e8bc7a9138e155773c: No bootstrap required, opened a new log
I20260811 17:19:50.119194 22993 ts_tablet_manager.cc:1397] T 3b56c5a5163f4b809b304bbf09c5d596 P 7c711e5e8faf44e8bc7a9138e155773c: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260811 17:19:50.119324 22991 tablet_bootstrap.cc:492] T 3b56c5a5163f4b809b304bbf09c5d596 P 1096fd23ccb24c7183fc39f31a1e00fc: No bootstrap required, opened a new log
I20260811 17:19:50.119405 22991 ts_tablet_manager.cc:1397] T 3b56c5a5163f4b809b304bbf09c5d596 P 1096fd23ccb24c7183fc39f31a1e00fc: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 17:19:50.119628 22995 tablet_bootstrap.cc:492] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8: No bootstrap required, opened a new log
I20260811 17:19:50.119743 22995 ts_tablet_manager.cc:1397] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260811 17:19:50.119797 22993 raft_consensus.cc:348] T 3b56c5a5163f4b809b304bbf09c5d596 P 7c711e5e8faf44e8bc7a9138e155773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:50.119837 22991 raft_consensus.cc:348] T 3b56c5a5163f4b809b304bbf09c5d596 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:50.119928 22993 raft_consensus.cc:374] T 3b56c5a5163f4b809b304bbf09c5d596 P 7c711e5e8faf44e8bc7a9138e155773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:50.119946 22991 raft_consensus.cc:374] T 3b56c5a5163f4b809b304bbf09c5d596 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:50.119964 22993 raft_consensus.cc:729] T 3b56c5a5163f4b809b304bbf09c5d596 P 7c711e5e8faf44e8bc7a9138e155773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c711e5e8faf44e8bc7a9138e155773c, State: Initialized, Role: FOLLOWER
I20260811 17:19:50.119974 22991 raft_consensus.cc:729] T 3b56c5a5163f4b809b304bbf09c5d596 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1096fd23ccb24c7183fc39f31a1e00fc, State: Initialized, Role: FOLLOWER
I20260811 17:19:50.120087 22991 consensus_queue.cc:260] T 3b56c5a5163f4b809b304bbf09c5d596 P 1096fd23ccb24c7183fc39f31a1e00fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:50.120096 22993 consensus_queue.cc:260] T 3b56c5a5163f4b809b304bbf09c5d596 P 7c711e5e8faf44e8bc7a9138e155773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:50.120172 22995 raft_consensus.cc:348] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:50.120276 22995 raft_consensus.cc:374] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:50.120306 22995 raft_consensus.cc:729] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fbf9ebada614af9b5679b3e602cecd8, State: Initialized, Role: FOLLOWER
I20260811 17:19:50.120342 22991 ts_tablet_manager.cc:1428] T 3b56c5a5163f4b809b304bbf09c5d596 P 1096fd23ccb24c7183fc39f31a1e00fc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 17:19:50.120371 22993 ts_tablet_manager.cc:1428] T 3b56c5a5163f4b809b304bbf09c5d596 P 7c711e5e8faf44e8bc7a9138e155773c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 17:19:50.120445 22995 consensus_queue.cc:260] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:50.120714 22995 ts_tablet_manager.cc:1428] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 17:19:50.240443 23063 raft_consensus.cc:482] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:19:50.240677 23063 raft_consensus.cc:504] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:50.241252 23063 leader_election.cc:288] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1096fd23ccb24c7183fc39f31a1e00fc (127.9.157.129:45711), 7c711e5e8faf44e8bc7a9138e155773c (127.9.157.131:46347)
I20260811 17:19:50.241842 22470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b56c5a5163f4b809b304bbf09c5d596" candidate_uuid: "9fbf9ebada614af9b5679b3e602cecd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" is_pre_election: true
I20260811 17:19:50.242039 22470 raft_consensus.cc:2454] T 3b56c5a5163f4b809b304bbf09c5d596 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fbf9ebada614af9b5679b3e602cecd8 in term 0.
I20260811 17:19:50.242095 22876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b56c5a5163f4b809b304bbf09c5d596" candidate_uuid: "9fbf9ebada614af9b5679b3e602cecd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c711e5e8faf44e8bc7a9138e155773c" is_pre_election: true
I20260811 17:19:50.242290 22876 raft_consensus.cc:2454] T 3b56c5a5163f4b809b304bbf09c5d596 P 7c711e5e8faf44e8bc7a9138e155773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fbf9ebada614af9b5679b3e602cecd8 in term 0.
I20260811 17:19:50.242408 22583 leader_election.cc:302] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1096fd23ccb24c7183fc39f31a1e00fc, 9fbf9ebada614af9b5679b3e602cecd8; no voters:
I20260811 17:19:50.242625 23063 raft_consensus.cc:2781] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:19:50.242690 23063 raft_consensus.cc:482] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:19:50.242766 23063 raft_consensus.cc:3037] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:50.244050 23063 raft_consensus.cc:504] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:50.244472 23063 leader_election.cc:288] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8 [CANDIDATE]: Term 1 election: Requested vote from peers 1096fd23ccb24c7183fc39f31a1e00fc (127.9.157.129:45711), 7c711e5e8faf44e8bc7a9138e155773c (127.9.157.131:46347)
I20260811 17:19:50.244864 22470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b56c5a5163f4b809b304bbf09c5d596" candidate_uuid: "9fbf9ebada614af9b5679b3e602cecd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1096fd23ccb24c7183fc39f31a1e00fc"
I20260811 17:19:50.244905 22876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b56c5a5163f4b809b304bbf09c5d596" candidate_uuid: "9fbf9ebada614af9b5679b3e602cecd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c711e5e8faf44e8bc7a9138e155773c"
I20260811 17:19:50.245018 22876 raft_consensus.cc:3037] T 3b56c5a5163f4b809b304bbf09c5d596 P 7c711e5e8faf44e8bc7a9138e155773c [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:50.245018 22470 raft_consensus.cc:3037] T 3b56c5a5163f4b809b304bbf09c5d596 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:50.246240 22876 raft_consensus.cc:2454] T 3b56c5a5163f4b809b304bbf09c5d596 P 7c711e5e8faf44e8bc7a9138e155773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fbf9ebada614af9b5679b3e602cecd8 in term 1.
I20260811 17:19:50.246240 22470 raft_consensus.cc:2454] T 3b56c5a5163f4b809b304bbf09c5d596 P 1096fd23ccb24c7183fc39f31a1e00fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fbf9ebada614af9b5679b3e602cecd8 in term 1.
I20260811 17:19:50.246658 22583 leader_election.cc:302] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1096fd23ccb24c7183fc39f31a1e00fc, 7c711e5e8faf44e8bc7a9138e155773c, 9fbf9ebada614af9b5679b3e602cecd8; no voters:
I20260811 17:19:50.246943 23063 raft_consensus.cc:2781] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:50.247054 23063 raft_consensus.cc:686] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8 [term 1 LEADER]: Becoming Leader. State: Replica: 9fbf9ebada614af9b5679b3e602cecd8, State: Running, Role: LEADER
I20260811 17:19:50.247210 23063 consensus_queue.cc:237] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:50.249089 22206 catalog_manager.cc:5495] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9fbf9ebada614af9b5679b3e602cecd8 (127.9.157.130). New cstate: current_term: 1 leader_uuid: "9fbf9ebada614af9b5679b3e602cecd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } health_report { overall_health: UNKNOWN } } }
I20260811 17:19:50.279991 22986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:50.276768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56202 (local address 127.9.157.130:40473)
0811 17:19:50.276919 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:50.276923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:50.276938 (+ 15us) server_negotiation.cc:407] Connection header received
0811 17:19:50.277075 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:50.277077 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:50.277128 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:50.277215 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:19:50.277516 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:50.278023 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:50.278899 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:50.279118 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:50.279225 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:50.279502 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:19:50.279577 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:19:50.279678 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:50.279780 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 17:19:50.279838 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 17:19:50.282581 22470 raft_consensus.cc:1261] T 3b56c5a5163f4b809b304bbf09c5d596 P 1096fd23ccb24c7183fc39f31a1e00fc [term 1 FOLLOWER]: Refusing update from remote peer 9fbf9ebada614af9b5679b3e602cecd8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:50.282593 22876 raft_consensus.cc:1261] T 3b56c5a5163f4b809b304bbf09c5d596 P 7c711e5e8faf44e8bc7a9138e155773c [term 1 FOLLOWER]: Refusing update from remote peer 9fbf9ebada614af9b5679b3e602cecd8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:50.283046 23053 consensus_queue.cc:1035] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:50.283226 23063 consensus_queue.cc:1035] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:50.294649 22206 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33044:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:19:50.294812 22206 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33044:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:19:50.297102 22206 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a15553ed49fb4a3d8511c2caf09b8aa6: Sending DeleteTablet for 3 replicas of tablet 3b56c5a5163f4b809b304bbf09c5d596
I20260811 17:19:50.297752 22652 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b56c5a5163f4b809b304bbf09c5d596 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:50 UTC) from {username='oryx', principal='oryx/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:56196
I20260811 17:19:50.297849 22851 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b56c5a5163f4b809b304bbf09c5d596 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:50 UTC) from {username='oryx', principal='oryx/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:34336
I20260811 17:19:50.297884 22434 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b56c5a5163f4b809b304bbf09c5d596 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:50 UTC) from {username='oryx', principal='oryx/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:48252
I20260811 17:19:50.297982 23035 tablet_replica.cc:330] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8: stopping tablet replica
I20260811 17:19:50.298040 23032 tablet_replica.cc:330] T 3b56c5a5163f4b809b304bbf09c5d596 P 7c711e5e8faf44e8bc7a9138e155773c: stopping tablet replica
I20260811 17:19:50.298064 23033 tablet_replica.cc:330] T 3b56c5a5163f4b809b304bbf09c5d596 P 1096fd23ccb24c7183fc39f31a1e00fc: stopping tablet replica
I20260811 17:19:50.298108 23035 raft_consensus.cc:2229] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:19:50.298154 23032 raft_consensus.cc:2229] T 3b56c5a5163f4b809b304bbf09c5d596 P 7c711e5e8faf44e8bc7a9138e155773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:19:50.298183 23033 raft_consensus.cc:2229] T 3b56c5a5163f4b809b304bbf09c5d596 P 1096fd23ccb24c7183fc39f31a1e00fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:19:50.298281 23032 raft_consensus.cc:2258] T 3b56c5a5163f4b809b304bbf09c5d596 P 7c711e5e8faf44e8bc7a9138e155773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:50.298313 23033 raft_consensus.cc:2258] T 3b56c5a5163f4b809b304bbf09c5d596 P 1096fd23ccb24c7183fc39f31a1e00fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:50.298456 23035 raft_consensus.cc:2258] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:50.299202 23035 ts_tablet_manager.cc:1905] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:50.299471 23033 ts_tablet_manager.cc:1905] T 3b56c5a5163f4b809b304bbf09c5d596 P 1096fd23ccb24c7183fc39f31a1e00fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:50.299757 23032 ts_tablet_manager.cc:1905] T 3b56c5a5163f4b809b304bbf09c5d596 P 7c711e5e8faf44e8bc7a9138e155773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:50.301963 23033 ts_tablet_manager.cc:1918] T 3b56c5a5163f4b809b304bbf09c5d596 P 1096fd23ccb24c7183fc39f31a1e00fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:19:50.301963 23035 ts_tablet_manager.cc:1918] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:19:50.302038 23035 log.cc:1196] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-1/wal/wals/3b56c5a5163f4b809b304bbf09c5d596
I20260811 17:19:50.302038 23033 log.cc:1196] T 3b56c5a5163f4b809b304bbf09c5d596 P 1096fd23ccb24c7183fc39f31a1e00fc: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-0/wal/wals/3b56c5a5163f4b809b304bbf09c5d596
I20260811 17:19:50.302129 23032 ts_tablet_manager.cc:1918] T 3b56c5a5163f4b809b304bbf09c5d596 P 7c711e5e8faf44e8bc7a9138e155773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:19:50.302203 23032 log.cc:1196] T 3b56c5a5163f4b809b304bbf09c5d596 P 7c711e5e8faf44e8bc7a9138e155773c: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-2/wal/wals/3b56c5a5163f4b809b304bbf09c5d596
I20260811 17:19:50.302318 23033 ts_tablet_manager.cc:1939] T 3b56c5a5163f4b809b304bbf09c5d596 P 1096fd23ccb24c7183fc39f31a1e00fc: Deleting consensus metadata
I20260811 17:19:50.302318 23035 ts_tablet_manager.cc:1939] T 3b56c5a5163f4b809b304bbf09c5d596 P 9fbf9ebada614af9b5679b3e602cecd8: Deleting consensus metadata
I20260811 17:19:50.302480 23032 ts_tablet_manager.cc:1939] T 3b56c5a5163f4b809b304bbf09c5d596 P 7c711e5e8faf44e8bc7a9138e155773c: Deleting consensus metadata
I20260811 17:19:50.303064 22183 catalog_manager.cc:4841] TS 9fbf9ebada614af9b5679b3e602cecd8 (127.9.157.130:40473): tablet 3b56c5a5163f4b809b304bbf09c5d596 (table test-table [id=5d20248272bb4c309bec8ca0ad64694c]) successfully deleted
I20260811 17:19:50.303141 22180 catalog_manager.cc:4841] TS 7c711e5e8faf44e8bc7a9138e155773c (127.9.157.131:46347): tablet 3b56c5a5163f4b809b304bbf09c5d596 (table test-table [id=5d20248272bb4c309bec8ca0ad64694c]) successfully deleted
I20260811 17:19:50.303679 22177 catalog_manager.cc:4841] TS 1096fd23ccb24c7183fc39f31a1e00fc (127.9.157.129:45711): tablet 3b56c5a5163f4b809b304bbf09c5d596 (table test-table [id=5d20248272bb4c309bec8ca0ad64694c]) successfully deleted
I20260811 17:19:50.304143 22206 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33044:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:19:50.304594 22206 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:19:50.310202 22434 tablet_service.cc:1467] Processing CreateTablet for tablet dca26668ea9e44f7b1d95086b204c1c8 (DEFAULT_TABLE table=test-table [id=0a98e0e49ea0497190bd8f8226ea35d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:50.310497 22652 tablet_service.cc:1467] Processing CreateTablet for tablet dca26668ea9e44f7b1d95086b204c1c8 (DEFAULT_TABLE table=test-table [id=0a98e0e49ea0497190bd8f8226ea35d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:50.310560 22434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dca26668ea9e44f7b1d95086b204c1c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:50.310871 22652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dca26668ea9e44f7b1d95086b204c1c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:50.313164 22991 tablet_bootstrap.cc:492] T dca26668ea9e44f7b1d95086b204c1c8 P 1096fd23ccb24c7183fc39f31a1e00fc: Bootstrap starting.
I20260811 17:19:50.313282 22995 tablet_bootstrap.cc:492] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8: Bootstrap starting.
I20260811 17:19:50.313525 22851 tablet_service.cc:1467] Processing CreateTablet for tablet dca26668ea9e44f7b1d95086b204c1c8 (DEFAULT_TABLE table=test-table [id=0a98e0e49ea0497190bd8f8226ea35d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:19:50.313805 22851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dca26668ea9e44f7b1d95086b204c1c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:50.314142 22991 tablet_bootstrap.cc:654] T dca26668ea9e44f7b1d95086b204c1c8 P 1096fd23ccb24c7183fc39f31a1e00fc: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:50.314142 22995 tablet_bootstrap.cc:654] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:50.315601 22991 tablet_bootstrap.cc:492] T dca26668ea9e44f7b1d95086b204c1c8 P 1096fd23ccb24c7183fc39f31a1e00fc: No bootstrap required, opened a new log
I20260811 17:19:50.315739 22991 ts_tablet_manager.cc:1397] T dca26668ea9e44f7b1d95086b204c1c8 P 1096fd23ccb24c7183fc39f31a1e00fc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 17:19:50.315932 22995 tablet_bootstrap.cc:492] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8: No bootstrap required, opened a new log
I20260811 17:19:50.316009 22995 ts_tablet_manager.cc:1397] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 17:19:50.316306 22991 raft_consensus.cc:348] T dca26668ea9e44f7b1d95086b204c1c8 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:50.316385 22993 tablet_bootstrap.cc:492] T dca26668ea9e44f7b1d95086b204c1c8 P 7c711e5e8faf44e8bc7a9138e155773c: Bootstrap starting.
I20260811 17:19:50.316438 22991 raft_consensus.cc:374] T dca26668ea9e44f7b1d95086b204c1c8 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:50.316399 22995 raft_consensus.cc:348] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:50.316475 22991 raft_consensus.cc:729] T dca26668ea9e44f7b1d95086b204c1c8 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1096fd23ccb24c7183fc39f31a1e00fc, State: Initialized, Role: FOLLOWER
I20260811 17:19:50.316512 22995 raft_consensus.cc:374] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:50.316548 22995 raft_consensus.cc:729] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fbf9ebada614af9b5679b3e602cecd8, State: Initialized, Role: FOLLOWER
I20260811 17:19:50.316622 22991 consensus_queue.cc:260] T dca26668ea9e44f7b1d95086b204c1c8 P 1096fd23ccb24c7183fc39f31a1e00fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:50.316668 22995 consensus_queue.cc:260] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:50.316960 22995 ts_tablet_manager.cc:1428] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 17:19:50.317306 22993 tablet_bootstrap.cc:654] T dca26668ea9e44f7b1d95086b204c1c8 P 7c711e5e8faf44e8bc7a9138e155773c: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:50.317859 22991 ts_tablet_manager.cc:1428] T dca26668ea9e44f7b1d95086b204c1c8 P 1096fd23ccb24c7183fc39f31a1e00fc: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 17:19:50.318998 22993 tablet_bootstrap.cc:492] T dca26668ea9e44f7b1d95086b204c1c8 P 7c711e5e8faf44e8bc7a9138e155773c: No bootstrap required, opened a new log
I20260811 17:19:50.319080 22993 ts_tablet_manager.cc:1397] T dca26668ea9e44f7b1d95086b204c1c8 P 7c711e5e8faf44e8bc7a9138e155773c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 17:19:50.319506 22993 raft_consensus.cc:348] T dca26668ea9e44f7b1d95086b204c1c8 P 7c711e5e8faf44e8bc7a9138e155773c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:50.319612 22993 raft_consensus.cc:374] T dca26668ea9e44f7b1d95086b204c1c8 P 7c711e5e8faf44e8bc7a9138e155773c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:50.319669 22993 raft_consensus.cc:729] T dca26668ea9e44f7b1d95086b204c1c8 P 7c711e5e8faf44e8bc7a9138e155773c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c711e5e8faf44e8bc7a9138e155773c, State: Initialized, Role: FOLLOWER
I20260811 17:19:50.319788 22993 consensus_queue.cc:260] T dca26668ea9e44f7b1d95086b204c1c8 P 7c711e5e8faf44e8bc7a9138e155773c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:50.320031 22993 ts_tablet_manager.cc:1428] T dca26668ea9e44f7b1d95086b204c1c8 P 7c711e5e8faf44e8bc7a9138e155773c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 17:19:50.373117 23063 raft_consensus.cc:482] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:19:50.373349 23063 raft_consensus.cc:504] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:50.373831 23063 leader_election.cc:288] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1096fd23ccb24c7183fc39f31a1e00fc (127.9.157.129:45711), 7c711e5e8faf44e8bc7a9138e155773c (127.9.157.131:46347)
I20260811 17:19:50.374295 22470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dca26668ea9e44f7b1d95086b204c1c8" candidate_uuid: "9fbf9ebada614af9b5679b3e602cecd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" is_pre_election: true
I20260811 17:19:50.374342 22876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dca26668ea9e44f7b1d95086b204c1c8" candidate_uuid: "9fbf9ebada614af9b5679b3e602cecd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c711e5e8faf44e8bc7a9138e155773c" is_pre_election: true
I20260811 17:19:50.374480 22470 raft_consensus.cc:2454] T dca26668ea9e44f7b1d95086b204c1c8 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fbf9ebada614af9b5679b3e602cecd8 in term 0.
I20260811 17:19:50.374557 22876 raft_consensus.cc:2454] T dca26668ea9e44f7b1d95086b204c1c8 P 7c711e5e8faf44e8bc7a9138e155773c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fbf9ebada614af9b5679b3e602cecd8 in term 0.
I20260811 17:19:50.374878 22583 leader_election.cc:302] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1096fd23ccb24c7183fc39f31a1e00fc, 9fbf9ebada614af9b5679b3e602cecd8; no voters:
I20260811 17:19:50.375080 23063 raft_consensus.cc:2781] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:19:50.375154 23063 raft_consensus.cc:482] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:19:50.375212 23063 raft_consensus.cc:3037] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:50.376531 23063 raft_consensus.cc:504] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:50.377017 23063 leader_election.cc:288] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8 [CANDIDATE]: Term 1 election: Requested vote from peers 1096fd23ccb24c7183fc39f31a1e00fc (127.9.157.129:45711), 7c711e5e8faf44e8bc7a9138e155773c (127.9.157.131:46347)
I20260811 17:19:50.377446 22470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dca26668ea9e44f7b1d95086b204c1c8" candidate_uuid: "9fbf9ebada614af9b5679b3e602cecd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1096fd23ccb24c7183fc39f31a1e00fc"
I20260811 17:19:50.377488 22876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dca26668ea9e44f7b1d95086b204c1c8" candidate_uuid: "9fbf9ebada614af9b5679b3e602cecd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c711e5e8faf44e8bc7a9138e155773c"
I20260811 17:19:50.377597 22470 raft_consensus.cc:3037] T dca26668ea9e44f7b1d95086b204c1c8 P 1096fd23ccb24c7183fc39f31a1e00fc [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:50.377650 22876 raft_consensus.cc:3037] T dca26668ea9e44f7b1d95086b204c1c8 P 7c711e5e8faf44e8bc7a9138e155773c [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:50.378623 22876 raft_consensus.cc:2454] T dca26668ea9e44f7b1d95086b204c1c8 P 7c711e5e8faf44e8bc7a9138e155773c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fbf9ebada614af9b5679b3e602cecd8 in term 1.
I20260811 17:19:50.378957 22470 raft_consensus.cc:2454] T dca26668ea9e44f7b1d95086b204c1c8 P 1096fd23ccb24c7183fc39f31a1e00fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fbf9ebada614af9b5679b3e602cecd8 in term 1.
I20260811 17:19:50.379005 22586 leader_election.cc:302] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c711e5e8faf44e8bc7a9138e155773c, 9fbf9ebada614af9b5679b3e602cecd8; no voters:
I20260811 17:19:50.379292 23063 raft_consensus.cc:2781] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:50.379406 23063 raft_consensus.cc:686] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8 [term 1 LEADER]: Becoming Leader. State: Replica: 9fbf9ebada614af9b5679b3e602cecd8, State: Running, Role: LEADER
I20260811 17:19:50.379568 23063 consensus_queue.cc:237] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } }
I20260811 17:19:50.381410 22203 catalog_manager.cc:5495] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9fbf9ebada614af9b5679b3e602cecd8 (127.9.157.130). New cstate: current_term: 1 leader_uuid: "9fbf9ebada614af9b5679b3e602cecd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9fbf9ebada614af9b5679b3e602cecd8" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 } health_report { overall_health: UNKNOWN } } }
I20260811 17:19:50.407541 23072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:50.403468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:40473 (local address 127.9.157.130:38293)
0811 17:19:50.403612 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:50.403629 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:50.403813 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:50.404146 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:50.404149 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:50.404155 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 17:19:50.404227 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:50.404232 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:50.405260 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:50.405263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:50.406216 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:50.406227 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:50.406871 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:50.406874 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:50.406944 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:50.406970 (+ 26us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 17:19:50.407219 (+ 249us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 17:19:50.407222 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 17:19:50.407272 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 17:19:50.407369 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 17:19:50.408039 22986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:50.403578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:38293 (local address 127.9.157.130:40473)
0811 17:19:50.403700 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:50.403709 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:50.403723 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:19:50.403918 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:50.403921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:50.403971 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:50.404043 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:19:50.404322 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:50.405145 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:50.406383 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:50.406712 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:50.406801 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:50.407060 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:19:50.407144 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:19:50.407196 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:50.407883 (+ 687us) server_negotiation.cc:299] Negotiation successful
0811 17:19:50.407932 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 17:19:50.415233 22876 raft_consensus.cc:1261] T dca26668ea9e44f7b1d95086b204c1c8 P 7c711e5e8faf44e8bc7a9138e155773c [term 1 FOLLOWER]: Refusing update from remote peer 9fbf9ebada614af9b5679b3e602cecd8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:50.415221 22470 raft_consensus.cc:1261] T dca26668ea9e44f7b1d95086b204c1c8 P 1096fd23ccb24c7183fc39f31a1e00fc [term 1 FOLLOWER]: Refusing update from remote peer 9fbf9ebada614af9b5679b3e602cecd8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:19:50.415673 23063 consensus_queue.cc:1035] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c711e5e8faf44e8bc7a9138e155773c" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 46347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:50.415838 23053 consensus_queue.cc:1035] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1096fd23ccb24c7183fc39f31a1e00fc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:19:50.452957 22206 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33044:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:19:50.453138 22206 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33044:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:19:50.455433 22206 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a15553ed49fb4a3d8511c2caf09b8aa6: Sending DeleteTablet for 3 replicas of tablet dca26668ea9e44f7b1d95086b204c1c8
I20260811 17:19:50.456063 22434 tablet_service.cc:1514] Processing DeleteTablet for tablet dca26668ea9e44f7b1d95086b204c1c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:50 UTC) from {username='oryx', principal='oryx/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:48252
I20260811 17:19:50.456247 22651 tablet_service.cc:1514] Processing DeleteTablet for tablet dca26668ea9e44f7b1d95086b204c1c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:50 UTC) from {username='oryx', principal='oryx/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:56196
I20260811 17:19:50.456290 23033 tablet_replica.cc:330] T dca26668ea9e44f7b1d95086b204c1c8 P 1096fd23ccb24c7183fc39f31a1e00fc: stopping tablet replica
I20260811 17:19:50.456264 22851 tablet_service.cc:1514] Processing DeleteTablet for tablet dca26668ea9e44f7b1d95086b204c1c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:19:50 UTC) from {username='oryx', principal='oryx/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:34336
I20260811 17:19:50.456418 23035 tablet_replica.cc:330] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8: stopping tablet replica
I20260811 17:19:50.456429 23033 raft_consensus.cc:2229] T dca26668ea9e44f7b1d95086b204c1c8 P 1096fd23ccb24c7183fc39f31a1e00fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:19:50.456516 23035 raft_consensus.cc:2229] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:19:50.456563 23033 raft_consensus.cc:2258] T dca26668ea9e44f7b1d95086b204c1c8 P 1096fd23ccb24c7183fc39f31a1e00fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:50.456743 23032 tablet_replica.cc:330] T dca26668ea9e44f7b1d95086b204c1c8 P 7c711e5e8faf44e8bc7a9138e155773c: stopping tablet replica
I20260811 17:19:50.456727 23035 raft_consensus.cc:2258] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:50.456864 23032 raft_consensus.cc:2229] T dca26668ea9e44f7b1d95086b204c1c8 P 7c711e5e8faf44e8bc7a9138e155773c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:19:50.456982 23032 raft_consensus.cc:2258] T dca26668ea9e44f7b1d95086b204c1c8 P 7c711e5e8faf44e8bc7a9138e155773c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:19:50.457574 23035 ts_tablet_manager.cc:1905] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:50.457870 23032 ts_tablet_manager.cc:1905] T dca26668ea9e44f7b1d95086b204c1c8 P 7c711e5e8faf44e8bc7a9138e155773c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:50.457890 23033 ts_tablet_manager.cc:1905] T dca26668ea9e44f7b1d95086b204c1c8 P 1096fd23ccb24c7183fc39f31a1e00fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:19:50.460485 23033 ts_tablet_manager.cc:1918] T dca26668ea9e44f7b1d95086b204c1c8 P 1096fd23ccb24c7183fc39f31a1e00fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:19:50.460628 23033 log.cc:1196] T dca26668ea9e44f7b1d95086b204c1c8 P 1096fd23ccb24c7183fc39f31a1e00fc: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-0/wal/wals/dca26668ea9e44f7b1d95086b204c1c8
I20260811 17:19:50.460695 23032 ts_tablet_manager.cc:1918] T dca26668ea9e44f7b1d95086b204c1c8 P 7c711e5e8faf44e8bc7a9138e155773c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:19:50.460834 23032 log.cc:1196] T dca26668ea9e44f7b1d95086b204c1c8 P 7c711e5e8faf44e8bc7a9138e155773c: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-2/wal/wals/dca26668ea9e44f7b1d95086b204c1c8
I20260811 17:19:50.460973 23035 ts_tablet_manager.cc:1918] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 17:19:50.461002 23033 ts_tablet_manager.cc:1939] T dca26668ea9e44f7b1d95086b204c1c8 P 1096fd23ccb24c7183fc39f31a1e00fc: Deleting consensus metadata
I20260811 17:19:50.461125 23035 log.cc:1196] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786468743878666-9846-0/minicluster-data/ts-1/wal/wals/dca26668ea9e44f7b1d95086b204c1c8
I20260811 17:19:50.461138 23032 ts_tablet_manager.cc:1939] T dca26668ea9e44f7b1d95086b204c1c8 P 7c711e5e8faf44e8bc7a9138e155773c: Deleting consensus metadata
I20260811 17:19:50.461450 23035 ts_tablet_manager.cc:1939] T dca26668ea9e44f7b1d95086b204c1c8 P 9fbf9ebada614af9b5679b3e602cecd8: Deleting consensus metadata
I20260811 17:19:50.461838 22180 catalog_manager.cc:4841] TS 7c711e5e8faf44e8bc7a9138e155773c (127.9.157.131:46347): tablet dca26668ea9e44f7b1d95086b204c1c8 (table test-table [id=0a98e0e49ea0497190bd8f8226ea35d3]) successfully deleted
I20260811 17:19:50.461838 22177 catalog_manager.cc:4841] TS 1096fd23ccb24c7183fc39f31a1e00fc (127.9.157.129:45711): tablet dca26668ea9e44f7b1d95086b204c1c8 (table test-table [id=0a98e0e49ea0497190bd8f8226ea35d3]) successfully deleted
I20260811 17:19:50.462481 22183 catalog_manager.cc:4841] TS 9fbf9ebada614af9b5679b3e602cecd8 (127.9.157.130:40473): tablet dca26668ea9e44f7b1d95086b204c1c8 (table test-table [id=0a98e0e49ea0497190bd8f8226ea35d3]) successfully deleted
I20260811 17:19:50.463246 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 22326
I20260811 17:19:50.482071 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 22561
I20260811 17:19:50.498993 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 22759
I20260811 17:19:50.517216 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 22147
2026-08-11T17:19:50Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3584 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:19:50 dist-test-slave-mjjr krb5kdc[23131](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:19:50 dist-test-slave-mjjr krb5kdc[23131](info): set up 2 sockets
Aug 11 17:19:50 dist-test-slave-mjjr krb5kdc[23131](info): commencing operation
krb5kdc: starting...
W20260811 17:19:51.786250 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:19:51 dist-test-slave-mjjr krb5kdc[23131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:19:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:19:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:19:51.924924 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:42489
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:41369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:42489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:52.043736 23147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:52.044095 23147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:52.044181 23147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:52.048552 23147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:19:52.048658 23147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:52.048693 23147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:52.048734 23147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:19:52.048772 23147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:19:52.054404 23147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:41369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:42489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:42489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:52.055778 23147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:52.056908 23147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:52.066169 23155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:52.066185 23157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:52.066258 23147 server_base.cc:1034] running on GCE node
W20260811 17:19:52.066259 23153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:52.067235 23147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:52.068444 23147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:52.069612 23147 hybrid_clock.cc:648] HybridClock initialized: now 1786468792069577 us; error 49 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T17:19:52Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1562 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:19:52 dist-test-slave-mjjr krb5kdc[23164](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:19:52 dist-test-slave-mjjr krb5kdc[23164](info): set up 2 sockets
Aug 11 17:19:52 dist-test-slave-mjjr krb5kdc[23164](info): commencing operation
krb5kdc: starting...
W20260811 17:19:53.340870 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:19:53 dist-test-slave-mjjr krb5kdc[23164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:19:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:19:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:19:53.478253 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:42453
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:45027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:42453
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:53.604771 23188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:53.605158 23188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:53.605255 23188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:53.609613 23188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:19:53.609745 23188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:53.609790 23188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:53.609838 23188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:19:53.609882 23188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:19:53.615711 23188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:45027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:42453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:42453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:53.617059 23188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:53.618250 23188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:19:53.627758 23188 server_base.cc:1034] running on GCE node
W20260811 17:19:53.627981 23193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:53.628026 23200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:53.628006 23194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:53.628756 23188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:53.630019 23188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:53.631239 23188 hybrid_clock.cc:648] HybridClock initialized: now 1786468793631213 us; error 53 us; skew 500 ppm
Aug 11 17:19:53 dist-test-slave-mjjr krb5kdc[23164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468793, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:53.634208 23188 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:19:53.635214 23188 webserver.cc:460] Webserver started at http://127.9.157.190:38563/ using document root <none> and password file <none>
I20260811 17:19:53.635898 23188 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:53.635995 23188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:53.636247 23188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:53.638445 23188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "25c03710d9b0431889361cf19ea42a06"
format_stamp: "Formatted at 2026-08-11 17:19:53 on dist-test-slave-mjjr"
server_key: "2850dbba0fd3ffd7de2fbffdd3203556"
server_key_iv: "1e0b993436a43107a6e6bf2a5c949e4d"
server_key_version: "encryptionkey@0"
I20260811 17:19:53.639079 23188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "25c03710d9b0431889361cf19ea42a06"
format_stamp: "Formatted at 2026-08-11 17:19:53 on dist-test-slave-mjjr"
server_key: "2850dbba0fd3ffd7de2fbffdd3203556"
server_key_iv: "1e0b993436a43107a6e6bf2a5c949e4d"
server_key_version: "encryptionkey@0"
I20260811 17:19:53.643250 23188 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:19:53.646024 23212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:53.647460 23188 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:19:53.647701 23188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "25c03710d9b0431889361cf19ea42a06"
format_stamp: "Formatted at 2026-08-11 17:19:53 on dist-test-slave-mjjr"
server_key: "2850dbba0fd3ffd7de2fbffdd3203556"
server_key_iv: "1e0b993436a43107a6e6bf2a5c949e4d"
server_key_version: "encryptionkey@0"
I20260811 17:19:53.647842 23188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:53.674871 23188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:53.678326 23188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:53.678577 23188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:53.686491 23188 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:42453
I20260811 17:19:53.686510 23284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:42453 every 8 connection(s)
I20260811 17:19:53.687578 23188 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:19:53.690975 23285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:19:53.697264 23285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06: Bootstrap starting.
I20260811 17:19:53.697685 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 23188
I20260811 17:19:53.697826 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:19:53.698149 9846 external_mini_cluster.cc:1428] Setting key 027af19025f9d5fdf40595d7f90a1f7c
I20260811 17:19:53.700377 23285 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06: Neither blocks nor log segments found. Creating new log.
I20260811 17:19:53.701185 23285 log.cc:824] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:53.703195 23285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06: No bootstrap required, opened a new log
I20260811 17:19:53.707170 23285 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c03710d9b0431889361cf19ea42a06" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42453 } }
I20260811 17:19:53.707497 23285 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:19:53.707562 23285 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25c03710d9b0431889361cf19ea42a06, State: Initialized, Role: FOLLOWER
Aug 11 17:19:53 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:53.708148 23285 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c03710d9b0431889361cf19ea42a06" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42453 } }
I20260811 17:19:53.708346 23285 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:19:53.708463 23285 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:19:53.708601 23285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:19:53.709882 23285 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c03710d9b0431889361cf19ea42a06" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42453 } }
I20260811 17:19:53.710261 23285 leader_election.cc:302] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25c03710d9b0431889361cf19ea42a06; no voters:
I20260811 17:19:53.710639 23285 leader_election.cc:288] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:19:53.710752 23295 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:19:53.711001 23295 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [term 1 LEADER]: Becoming Leader. State: Replica: 25c03710d9b0431889361cf19ea42a06, State: Running, Role: LEADER
I20260811 17:19:53.711380 23295 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c03710d9b0431889361cf19ea42a06" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42453 } }
I20260811 17:19:53.712196 23285 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:19:53.713335 23298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25c03710d9b0431889361cf19ea42a06. Latest consensus state: current_term: 1 leader_uuid: "25c03710d9b0431889361cf19ea42a06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c03710d9b0431889361cf19ea42a06" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42453 } } }
I20260811 17:19:53.713480 23298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:53.713796 23296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25c03710d9b0431889361cf19ea42a06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c03710d9b0431889361cf19ea42a06" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42453 } } }
I20260811 17:19:53.713878 23296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:53.713933 23309 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:19:53.714480 23293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:53.699725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54272 (local address 127.9.157.190:42453)
0811 17:19:53.700240 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:53.700258 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:53.700299 (+ 41us) server_negotiation.cc:407] Connection header received
0811 17:19:53.701056 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:53.701075 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:53.701497 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:53.701856 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:53.702726 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:53.703896 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:53.704778 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:53.705275 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:53.705462 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:53.708456 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:53.708481 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:53.708502 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:53.708544 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:53.710499 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:53.711094 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:53.711100 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:53.711107 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:53.711248 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:53.711598 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:53.711601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:53.711603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:53.712189 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:53.712377 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:53.712914 (+ 537us) server_negotiation.cc:299] Negotiation successful
0811 17:19:53.713298 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":158,"threads_started":1}
I20260811 17:19:53.718367 23309 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:19:53.724318 23309 catalog_manager.cc:1347] Generated new cluster ID: 89f8796892ba431fa05551ab23871225
I20260811 17:19:53.724414 23309 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:19:53.751667 23309 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:19:53.753099 23309 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:19:53.768436 23309 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06: Generated new TSK 0
I20260811 17:19:53.769418 23309 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
I20260811 17:19:53.803949 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:0
--local_ip_for_outbound_sockets=127.9.157.129
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:42453
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:45027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:53.932528 23322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:53.932821 23322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:53.932868 23322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:53.937096 23322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:53.937181 23322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:53.937242 23322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:19:53.942796 23322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:45027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:42453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:53.944024 23322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:53.945070 23322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:53.955452 23328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:53.955474 23332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:53.955561 23322 server_base.cc:1034] running on GCE node
W20260811 17:19:53.955474 23329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:53.956194 23322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:53.956866 23322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:53.958033 23322 hybrid_clock.cc:648] HybridClock initialized: now 1786468793957997 us; error 52 us; skew 500 ppm
Aug 11 17:19:53 dist-test-slave-mjjr krb5kdc[23164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468793, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:53.961027 23322 init.cc:377] Logged in from keytab as kudu/127.9.157.129@KRBTEST.COM (short username kudu)
I20260811 17:19:53.961992 23322 webserver.cc:460] Webserver started at http://127.9.157.129:43095/ using document root <none> and password file <none>
I20260811 17:19:53.962553 23322 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:53.962615 23322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:53.962793 23322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:53.965031 23322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/data/instance:
uuid: "16a89ed0dbbd4e809d1a273c3bacb6e7"
format_stamp: "Formatted at 2026-08-11 17:19:53 on dist-test-slave-mjjr"
server_key: "0d26be9b4e910c8028bbf1e558f1d5be"
server_key_iv: "66a361ea5ec972935aa0076969c64e23"
server_key_version: "encryptionkey@0"
I20260811 17:19:53.965597 23322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance:
uuid: "16a89ed0dbbd4e809d1a273c3bacb6e7"
format_stamp: "Formatted at 2026-08-11 17:19:53 on dist-test-slave-mjjr"
server_key: "0d26be9b4e910c8028bbf1e558f1d5be"
server_key_iv: "66a361ea5ec972935aa0076969c64e23"
server_key_version: "encryptionkey@0"
I20260811 17:19:53.969959 23322 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 17:19:53.972702 23344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:53.974083 23322 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 17:19:53.974251 23322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "16a89ed0dbbd4e809d1a273c3bacb6e7"
format_stamp: "Formatted at 2026-08-11 17:19:53 on dist-test-slave-mjjr"
server_key: "0d26be9b4e910c8028bbf1e558f1d5be"
server_key_iv: "66a361ea5ec972935aa0076969c64e23"
server_key_version: "encryptionkey@0"
I20260811 17:19:53.974383 23322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:53.995803 23322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:53.999408 23322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:53.999693 23322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:54.000366 23322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:54.001272 23322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:54.001348 23322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:54.001432 23322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:54.001485 23322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:54.011233 23322 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:40999
I20260811 17:19:54.011260 23503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:40999 every 8 connection(s)
I20260811 17:19:54.012363 23322 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:19:54.012715 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 23322
I20260811 17:19:54.012897 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance
I20260811 17:19:54.013262 9846 external_mini_cluster.cc:1428] Setting key 270c94b164bb26aa0291dbcf72dbff94
Aug 11 17:19:54 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468793, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.157.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.130@KRBTEST.COM" created.
I20260811 17:19:54.029559 23293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:54.014606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:56125 (local address 127.9.157.190:42453)
0811 17:19:54.014775 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:54.014782 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:54.015988 (+ 1206us) server_negotiation.cc:407] Connection header received
0811 17:19:54.017890 (+ 1902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:54.017893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:54.017952 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:54.018068 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:54.019741 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:54.020293 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:54.022259 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:54.022494 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:54.022582 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:54.025642 (+ 3060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:54.025646 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:54.025648 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:54.025674 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:54.027151 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:54.027817 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:54.027821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:54.027824 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:54.027941 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:54.028369 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:54.028372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:54.028374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:54.028576 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:54.028688 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:54.029238 (+ 550us) server_negotiation.cc:299] Negotiation successful
0811 17:19:54.029378 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 17:19:54.030480 23507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:54.015184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.129:56125)
0811 17:19:54.015797 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:54.015865 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:54.017654 (+ 1789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:54.018262 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:54.018270 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:54.018744 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:54.019497 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:54.019520 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:54.020446 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:54.020453 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:54.022103 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:54.022120 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:54.022684 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:54.022689 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:54.022938 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:54.023618 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:54.023670 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:54.025451 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:54.027285 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:54.027292 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:54.027309 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:54.027707 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:54.028049 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:54.028052 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:54.028055 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:54.028261 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:54.028689 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:54.028698 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:54.028982 (+ 284us) client_negotiation.cc:769] Sending connection context
0811 17:19:54.029180 (+ 198us) client_negotiation.cc:240] Negotiation successful
0811 17:19:54.029560 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":141,"threads_started":1}
I20260811 17:19:54.031948 23505 heartbeater.cc:347] Connected to a master server at 127.9.157.190:42453
I20260811 17:19:54.032232 23505 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:54.032790 23505 heartbeater.cc:511] Master 127.9.157.190:42453 requested a full tablet report, sending...
I20260811 17:19:54.034703 23234 ts_manager.cc:194] Registered new tserver with Master: 16a89ed0dbbd4e809d1a273c3bacb6e7 (127.9.157.129:40999)
I20260811 17:19:54.036484 23234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.129@KRBTEST.COM'} at 127.9.157.129:56125
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
I20260811 17:19:54.044926 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:0
--local_ip_for_outbound_sockets=127.9.157.130
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:42453
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:45027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:54.171984 23512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:54.172283 23512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:54.172330 23512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:54.176471 23512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:54.176544 23512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:54.176607 23512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:19:54.182173 23512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:45027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:42453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:54.183296 23512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:54.184324 23512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:54.194572 23522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:54.194748 23512 server_base.cc:1034] running on GCE node
W20260811 17:19:54.194571 23519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:54.194963 23518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:54.195447 23512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:54.196148 23512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:54.197337 23512 hybrid_clock.cc:648] HybridClock initialized: now 1786468794197311 us; error 31 us; skew 500 ppm
Aug 11 17:19:54 dist-test-slave-mjjr krb5kdc[23164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468794, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:54.200454 23512 init.cc:377] Logged in from keytab as kudu/127.9.157.130@KRBTEST.COM (short username kudu)
I20260811 17:19:54.201392 23512 webserver.cc:460] Webserver started at http://127.9.157.130:39085/ using document root <none> and password file <none>
I20260811 17:19:54.202006 23512 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:54.202119 23512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:54.202363 23512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:54.204613 23512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/data/instance:
uuid: "b9204efe6e0a4102b3779516e954af0e"
format_stamp: "Formatted at 2026-08-11 17:19:54 on dist-test-slave-mjjr"
server_key: "bfbf3d737513c9f4681c441e966f79a5"
server_key_iv: "a292f4a21a21101f921d1921d135de02"
server_key_version: "encryptionkey@0"
I20260811 17:19:54.205276 23512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance:
uuid: "b9204efe6e0a4102b3779516e954af0e"
format_stamp: "Formatted at 2026-08-11 17:19:54 on dist-test-slave-mjjr"
server_key: "bfbf3d737513c9f4681c441e966f79a5"
server_key_iv: "a292f4a21a21101f921d1921d135de02"
server_key_version: "encryptionkey@0"
I20260811 17:19:54.209307 23512 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:19:54.212066 23533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:54.213645 23512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 17:19:54.213789 23512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "b9204efe6e0a4102b3779516e954af0e"
format_stamp: "Formatted at 2026-08-11 17:19:54 on dist-test-slave-mjjr"
server_key: "bfbf3d737513c9f4681c441e966f79a5"
server_key_iv: "a292f4a21a21101f921d1921d135de02"
server_key_version: "encryptionkey@0"
I20260811 17:19:54.213923 23512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:54.238181 23512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:54.241389 23512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:54.241624 23512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:54.242285 23512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:54.243184 23512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:54.243255 23512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:54.243325 23512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:54.243373 23512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:54.252954 23512 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:35691
I20260811 17:19:54.252976 23700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:35691 every 8 connection(s)
I20260811 17:19:54.253973 23512 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:19:54.263559 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 23512
I20260811 17:19:54.263738 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance
Aug 11 17:19:54 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468794, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:54.264056 9846 external_mini_cluster.cc:1428] Setting key 959517595f39e3de42366e34bc45538f
I20260811 17:19:54.268630 23293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:54.255985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:33207 (local address 127.9.157.190:42453)
0811 17:19:54.256166 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:54.256171 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:54.257048 (+ 877us) server_negotiation.cc:407] Connection header received
0811 17:19:54.258333 (+ 1285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:54.258336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:54.258386 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:54.258468 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:54.259734 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:54.260266 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:54.261395 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:54.261619 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:54.261696 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:54.264511 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:54.264514 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:54.264516 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:54.264544 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:54.266270 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:54.266887 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:54.266890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:54.266892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:54.266998 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:54.267397 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:54.267400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:54.267402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:54.267607 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:54.267760 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:54.268319 (+ 559us) server_negotiation.cc:299] Negotiation successful
0811 17:19:54.268456 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 17:19:54.269732 23704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:54.256401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.130:33207)
0811 17:19:54.256888 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:54.256932 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:54.258133 (+ 1201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:54.258644 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:54.258651 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:54.259053 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:54.259554 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:54.259567 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:54.260385 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:54.260388 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:54.261275 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:54.261284 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:54.261759 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:54.261761 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:54.261911 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:54.262493 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:54.262510 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:54.264303 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:54.266423 (+ 2120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:54.266431 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:54.266469 (+ 38us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:54.266758 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:54.267099 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:54.267102 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:54.267104 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:54.267277 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:54.267762 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:54.267773 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:54.268067 (+ 294us) client_negotiation.cc:769] Sending connection context
0811 17:19:54.268278 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 17:19:54.268673 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":128,"threads_started":1}
I20260811 17:19:54.271133 23701 heartbeater.cc:347] Connected to a master server at 127.9.157.190:42453
I20260811 17:19:54.271427 23701 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:54.272117 23701 heartbeater.cc:511] Master 127.9.157.190:42453 requested a full tablet report, sending...
I20260811 17:19:54.273412 23234 ts_manager.cc:194] Registered new tserver with Master: b9204efe6e0a4102b3779516e954af0e (127.9.157.130:35691)
I20260811 17:19:54.274042 23234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.130@KRBTEST.COM'} at 127.9.157.130:33207
WARNING: no policy specified for kudu/127.9.157.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
I20260811 17:19:54.295110 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:0
--local_ip_for_outbound_sockets=127.9.157.131
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:42453
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:45027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:54.416376 23707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:54.416733 23707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:54.416816 23707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:54.421208 23707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:54.421344 23707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:54.421443 23707 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:19:54.427412 23707 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:45027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:42453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:54.428737 23707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:54.429904 23707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:19:54.440446 23707 server_base.cc:1034] running on GCE node
W20260811 17:19:54.440382 23714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:54.440376 23716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:54.440775 23718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:54.441241 23707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:54.441978 23707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:54.443181 23707 hybrid_clock.cc:648] HybridClock initialized: now 1786468794443144 us; error 49 us; skew 500 ppm
Aug 11 17:19:54 dist-test-slave-mjjr krb5kdc[23164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468794, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:54.446259 23707 init.cc:377] Logged in from keytab as kudu/127.9.157.131@KRBTEST.COM (short username kudu)
I20260811 17:19:54.447209 23707 webserver.cc:460] Webserver started at http://127.9.157.131:33929/ using document root <none> and password file <none>
I20260811 17:19:54.447878 23707 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:54.447978 23707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:54.448231 23707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:19:54.450371 23707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/data/instance:
uuid: "1353ceb901d040bb9881efb643f7abf5"
format_stamp: "Formatted at 2026-08-11 17:19:54 on dist-test-slave-mjjr"
server_key: "3660d0086f3bf0447ad5725f571d7e12"
server_key_iv: "f68bd3346b48ebe1854be93ee8ad6d02"
server_key_version: "encryptionkey@0"
I20260811 17:19:54.450963 23707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance:
uuid: "1353ceb901d040bb9881efb643f7abf5"
format_stamp: "Formatted at 2026-08-11 17:19:54 on dist-test-slave-mjjr"
server_key: "3660d0086f3bf0447ad5725f571d7e12"
server_key_iv: "f68bd3346b48ebe1854be93ee8ad6d02"
server_key_version: "encryptionkey@0"
I20260811 17:19:54.455156 23707 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:19:54.457902 23729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:54.459208 23707 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 17:19:54.459362 23707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "1353ceb901d040bb9881efb643f7abf5"
format_stamp: "Formatted at 2026-08-11 17:19:54 on dist-test-slave-mjjr"
server_key: "3660d0086f3bf0447ad5725f571d7e12"
server_key_iv: "f68bd3346b48ebe1854be93ee8ad6d02"
server_key_version: "encryptionkey@0"
I20260811 17:19:54.459491 23707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:54.474936 23707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:54.478080 23707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:54.478343 23707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:54.479009 23707 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:54.479976 23707 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:54.480046 23707 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:54.480122 23707 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:54.480175 23707 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:54.489609 23707 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:36527
I20260811 17:19:54.489661 23901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:36527 every 8 connection(s)
I20260811 17:19:54.490612 23707 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:19:54.493203 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 23707
I20260811 17:19:54.493363 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance
I20260811 17:19:54.493745 9846 external_mini_cluster.cc:1428] Setting key 1c4afa224511da6e50ff58757d375438
Aug 11 17:19:54 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468794, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:19:54.505501 23293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:54.492800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:42065 (local address 127.9.157.190:42453)
0811 17:19:54.492948 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:54.492952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:54.494449 (+ 1497us) server_negotiation.cc:407] Connection header received
0811 17:19:54.496274 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:54.496277 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:54.496325 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:54.496386 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:54.497558 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:54.498087 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:54.499189 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:54.499400 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:54.499479 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:54.502120 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:54.502123 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:54.502126 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:54.502151 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:54.503498 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:54.504074 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:54.504078 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:54.504079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:54.504155 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:54.504485 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:54.504487 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:54.504488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:54.504669 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:54.504743 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:54.505263 (+ 520us) server_negotiation.cc:299] Negotiation successful
0811 17:19:54.505370 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 17:19:54.506417 23907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:54.493294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.131:42065)
0811 17:19:54.494271 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:54.494343 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:54.496031 (+ 1688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:54.496536 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:54.496544 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:54.496918 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:54.497431 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:54.497444 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:54.498200 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:54.498203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:54.499081 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:54.499090 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:54.499539 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:54.499541 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:54.499762 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:54.500425 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:54.500442 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:54.501954 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:19:54.503642 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:54.503673 (+ 31us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:54.503692 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:54.503980 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:54.504239 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:19:54.504241 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:19:54.504242 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:19:54.504396 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:19:54.504754 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:19:54.504759 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:19:54.505037 (+ 278us) client_negotiation.cc:769] Sending connection context
0811 17:19:54.505197 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 17:19:54.505471 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"thread_start_us":177,"threads_started":1}
I20260811 17:19:54.507581 23904 heartbeater.cc:347] Connected to a master server at 127.9.157.190:42453
I20260811 17:19:54.507875 23904 heartbeater.cc:464] Registering TS with master...
I20260811 17:19:54.508410 23904 heartbeater.cc:511] Master 127.9.157.190:42453 requested a full tablet report, sending...
I20260811 17:19:54.509469 23234 ts_manager.cc:194] Registered new tserver with Master: 1353ceb901d040bb9881efb643f7abf5 (127.9.157.131:36527)
I20260811 17:19:54.510049 23234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.131@KRBTEST.COM'} at 127.9.157.131:42065
I20260811 17:19:54.520056 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.129.keytab.
Entry for principal oryx/127.9.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.129.keytab.
WARNING: no policy specified for oryx/127.9.157.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.157.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.130.keytab.
Entry for principal oryx/127.9.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.130.keytab.
WARNING: no policy specified for oryx/127.9.157.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.157.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.131.keytab.
Entry for principal oryx/127.9.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.131.keytab.
I20260811 17:19:54.602569 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 23322
I20260811 17:19:54.620936 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 23512
I20260811 17:19:54.638294 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 23707
I20260811 17:19:54.651325 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 23188
I20260811 17:19:54.667882 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:42453
--webserver_interface=127.9.157.190
--webserver_port=38563
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:45027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:42453
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:19:54.800994 23916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:54.801442 23916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:54.801543 23916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:54.811697 23916 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:19:54.811820 23916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:54.811863 23916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:54.811910 23916 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:19:54.811954 23916 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:19:54.824378 23916 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:45027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:42453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:42453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=38563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:54.826066 23916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:54.827172 23916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:54.836213 23926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:54.836236 23924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:54.836215 23923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:54.836524 23916 server_base.cc:1034] running on GCE node
I20260811 17:19:54.837231 23916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:54.838472 23916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:54.839706 23916 hybrid_clock.cc:648] HybridClock initialized: now 1786468794839687 us; error 33 us; skew 500 ppm
Aug 11 17:19:54 dist-test-slave-mjjr krb5kdc[23164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468794, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:54.842615 23916 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:19:54.843688 23916 webserver.cc:460] Webserver started at http://127.9.157.190:38563/ using document root <none> and password file <none>
I20260811 17:19:54.844319 23916 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:54.844441 23916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:54.848627 23916 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 17:19:54.851310 23935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:54.852748 23916 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 17:19:54.852934 23916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "25c03710d9b0431889361cf19ea42a06"
format_stamp: "Formatted at 2026-08-11 17:19:53 on dist-test-slave-mjjr"
server_key: "2850dbba0fd3ffd7de2fbffdd3203556"
server_key_iv: "1e0b993436a43107a6e6bf2a5c949e4d"
server_key_version: "encryptionkey@0"
I20260811 17:19:54.853525 23916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:54.894059 23916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:54.898217 23916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:54.898425 23916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:54.906960 23916 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:42453
I20260811 17:19:54.907013 24013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:42453 every 8 connection(s)
I20260811 17:19:54.908416 23916 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:19:54.910482 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 23916
I20260811 17:19:54.911036 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:40999
--local_ip_for_outbound_sockets=127.9.157.129
--tserver_master_addrs=127.9.157.190:42453
--webserver_port=43095
--webserver_interface=127.9.157.129
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:45027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.157.129
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.129.keytab with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:19:54.914453 24016 sys_catalog.cc:263] Verifying existing consensus state
I20260811 17:19:54.916581 24016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06: Bootstrap starting.
I20260811 17:19:54.935395 24016 log.cc:824] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06: Log is configured to *not* fsync() on all Append() calls
I20260811 17:19:54.942814 24016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 17:19:54.943306 24016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06: Bootstrap complete.
I20260811 17:19:54.948208 24016 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c03710d9b0431889361cf19ea42a06" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42453 } }
I20260811 17:19:54.948648 24016 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25c03710d9b0431889361cf19ea42a06, State: Initialized, Role: FOLLOWER
I20260811 17:19:54.949193 24016 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c03710d9b0431889361cf19ea42a06" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42453 } }
I20260811 17:19:54.949312 24016 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:19:54.949405 24016 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:19:54.949512 24016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [term 1 FOLLOWER]: Advancing to term 2
I20260811 17:19:54.951705 24016 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c03710d9b0431889361cf19ea42a06" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42453 } }
I20260811 17:19:54.952097 24016 leader_election.cc:302] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25c03710d9b0431889361cf19ea42a06; no voters:
I20260811 17:19:54.952479 24016 leader_election.cc:288] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 17:19:54.952581 24021 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 17:19:54.952804 24021 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [term 2 LEADER]: Becoming Leader. State: Replica: 25c03710d9b0431889361cf19ea42a06, State: Running, Role: LEADER
I20260811 17:19:54.953125 24021 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c03710d9b0431889361cf19ea42a06" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42453 } }
I20260811 17:19:54.953860 24016 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:19:54.954188 24021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25c03710d9b0431889361cf19ea42a06. Latest consensus state: current_term: 2 leader_uuid: "25c03710d9b0431889361cf19ea42a06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c03710d9b0431889361cf19ea42a06" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42453 } } }
I20260811 17:19:54.954319 24021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:54.954723 24025 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:19:54.954746 24022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "25c03710d9b0431889361cf19ea42a06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c03710d9b0431889361cf19ea42a06" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42453 } } }
I20260811 17:19:54.954998 24022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06 [sys.catalog]: This master's current role is: LEADER
I20260811 17:19:54.958611 24025 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:19:54.959408 24025 catalog_manager.cc:1259] Loaded cluster ID: 89f8796892ba431fa05551ab23871225
I20260811 17:19:54.959498 24025 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:19:54.961357 24025 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:19:54.961982 24025 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 25c03710d9b0431889361cf19ea42a06: Loaded TSK: 0
I20260811 17:19:54.963006 24025 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 17:19:55.047585 24018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:55.047999 24018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:55.048105 24018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:55.053176 24018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:55.053292 24018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:55.053397 24018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:19:55.059907 24018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:45027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.129.keytab
--principal=oryx/127.9.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:40999
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=43095
--tserver_master_addrs=127.9.157.190:42453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:55.061096 24018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:55.062218 24018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:55.074128 24053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:55.074430 24018 server_base.cc:1034] running on GCE node
W20260811 17:19:55.074136 24048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:55.074136 24049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:55.075065 24018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:55.076047 24018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:55.077270 24018 hybrid_clock.cc:648] HybridClock initialized: now 1786468795077242 us; error 39 us; skew 500 ppm
Aug 11 17:19:55 dist-test-slave-mjjr krb5kdc[23164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468795, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:55.080924 24018 init.cc:377] Logged in from keytab as oryx/127.9.157.129@KRBTEST.COM (short username oryx)
I20260811 17:19:55.081923 24018 webserver.cc:460] Webserver started at http://127.9.157.129:43095/ using document root <none> and password file <none>
I20260811 17:19:55.082528 24018 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:55.082607 24018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:55.086912 24018 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:19:55.089370 24061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:55.090816 24018 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:19:55.090984 24018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "16a89ed0dbbd4e809d1a273c3bacb6e7"
format_stamp: "Formatted at 2026-08-11 17:19:53 on dist-test-slave-mjjr"
server_key: "0d26be9b4e910c8028bbf1e558f1d5be"
server_key_iv: "66a361ea5ec972935aa0076969c64e23"
server_key_version: "encryptionkey@0"
I20260811 17:19:55.091570 24018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:55.113699 24018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:55.117359 24018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:55.117585 24018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:55.118259 24018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:55.119180 24018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:55.119256 24018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:55.119330 24018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:55.119381 24018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:55.129022 24018 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:40999
I20260811 17:19:55.129092 24227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:40999 every 8 connection(s)
I20260811 17:19:55.130218 24018 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:19:55.140290 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 24018
I20260811 17:19:55.140882 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:35691
--local_ip_for_outbound_sockets=127.9.157.130
--tserver_master_addrs=127.9.157.190:42453
--webserver_port=39085
--webserver_interface=127.9.157.130
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:45027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.157.130
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.130.keytab with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 17:19:55 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:19:55 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 17:19:55.144481 24230 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:55.132616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:58317 (local address 127.9.157.190:42453)
0811 17:19:55.133111 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:55.133128 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:55.133354 (+ 226us) server_negotiation.cc:407] Connection header received
0811 17:19:55.134909 (+ 1555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:55.134920 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:55.135221 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:55.135514 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:55.136729 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.137732 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.138880 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.139369 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.139494 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:55.143593 (+ 4099us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.129:58317: BlockingRecv error: recv got EOF from 127.9.157.129:58317 (error 108)
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":144,"threads_started":1}
I20260811 17:19:55.144491 24231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:55.132677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.129:58317)
0811 17:19:55.133170 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:55.133224 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:55.134531 (+ 1307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:55.135610 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:55.135617 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:55.136065 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:55.136553 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:55.136567 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.137869 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.137872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:55.138763 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:55.138773 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.139522 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.139525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:55.139752 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:55.140466 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:55.140489 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:55.143344 (+ 2855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":164,"threads_started":1}
W20260811 17:19:55.145095 24228 heartbeater.cc:644] Failed to heartbeat to 127.9.157.190:42453 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.157.190:42453: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Aug 11 17:19:55 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:19:55 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:19:55.152707 24231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:55.145629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.129:57703)
0811 17:19:55.145844 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:55.145862 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:55.145965 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:55.146245 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:55.146251 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:55.146495 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:55.146670 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:55.146684 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.147562 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.147568 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:55.148729 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:55.148746 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.149349 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.149355 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:55.149487 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:55.150148 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:55.150169 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:55.152506 (+ 2337us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129}
W20260811 17:19:55.152813 24230 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:55.145715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:57703 (local address 127.9.157.190:42453)
0811 17:19:55.145837 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:55.145841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:55.145925 (+ 84us) server_negotiation.cc:407] Connection header received
0811 17:19:55.146075 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:55.146077 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:55.146123 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:55.146189 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:55.146811 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.147424 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.148920 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.149175 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.149295 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:55.152674 (+ 3379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.129:57703: BlockingRecv error: recv got EOF from 127.9.157.129:57703 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 11 17:19:55 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:19:55 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:19:55.160351 24231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:55.153749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.129:43341)
0811 17:19:55.153905 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:55.153927 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:55.154028 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:55.154318 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:55.154320 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:55.154465 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:55.154551 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:55.154555 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.155785 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.155788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:55.156411 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:55.156423 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.157068 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.157071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:55.157138 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:55.157763 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:55.157780 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:55.160184 (+ 2404us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":73}
W20260811 17:19:55.160538 24230 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:55.153849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:43341 (local address 127.9.157.190:42453)
0811 17:19:55.154022 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:55.154030 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:55.154051 (+ 21us) server_negotiation.cc:407] Connection header received
0811 17:19:55.154103 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:55.154109 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:55.154177 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:55.154275 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:55.154679 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.155602 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.156534 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.156894 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.157055 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:55.160357 (+ 3302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.129:43341: BlockingRecv error: recv got EOF from 127.9.157.129:43341 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 17:19:55.160694 24228 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 17:19:55.268659 24233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:55.268976 24233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:55.269027 24233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:55.273530 24233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:55.273631 24233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:55.273697 24233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:19:55.279495 24233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:45027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.130.keytab
--principal=oryx/127.9.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:35691
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=39085
--tserver_master_addrs=127.9.157.190:42453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:55.280807 24233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:55.281966 24233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:55.292496 24239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:55.292510 24238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:55.292629 24233 server_base.cc:1034] running on GCE node
W20260811 17:19:55.292498 24241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:55.293305 24233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:55.294028 24233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:55.295221 24233 hybrid_clock.cc:648] HybridClock initialized: now 1786468795295203 us; error 32 us; skew 500 ppm
Aug 11 17:19:55 dist-test-slave-mjjr krb5kdc[23164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468795, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:55.298382 24233 init.cc:377] Logged in from keytab as oryx/127.9.157.130@KRBTEST.COM (short username oryx)
I20260811 17:19:55.299414 24233 webserver.cc:460] Webserver started at http://127.9.157.130:39085/ using document root <none> and password file <none>
I20260811 17:19:55.300146 24233 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:55.300274 24233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:55.304740 24233 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 17:19:55.307379 24249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:55.309019 24233 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:55.309216 24233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "b9204efe6e0a4102b3779516e954af0e"
format_stamp: "Formatted at 2026-08-11 17:19:54 on dist-test-slave-mjjr"
server_key: "bfbf3d737513c9f4681c441e966f79a5"
server_key_iv: "a292f4a21a21101f921d1921d135de02"
server_key_version: "encryptionkey@0"
I20260811 17:19:55.309880 24233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:55.321748 24233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:55.325546 24233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:55.325796 24233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:55.326547 24233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:55.327538 24233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:55.327616 24233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:55.327708 24233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:55.327754 24233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:55.338055 24233 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:35691
I20260811 17:19:55.338094 24421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:35691 every 8 connection(s)
I20260811 17:19:55.339247 24233 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:19:55.341004 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 24233
I20260811 17:19:55.341492 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:36527
--local_ip_for_outbound_sockets=127.9.157.131
--tserver_master_addrs=127.9.157.190:42453
--webserver_port=33929
--webserver_interface=127.9.157.131
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:45027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.157.131
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.131.keytab with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 17:19:55 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:19:55 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 17:19:55.354511 24230 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:55.341610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:45241 (local address 127.9.157.190:42453)
0811 17:19:55.341797 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:55.341802 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:55.342978 (+ 1176us) server_negotiation.cc:407] Connection header received
0811 17:19:55.344916 (+ 1938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:55.344919 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:55.344996 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:55.345106 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:55.346478 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.347343 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.348880 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.349335 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.349481 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:55.354318 (+ 4837us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.130:45241: BlockingRecv error: recv got EOF from 127.9.157.130:45241 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 17:19:55.355212 24424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:55.342129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.130:45241)
0811 17:19:55.342744 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:55.342813 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:55.344626 (+ 1813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:55.345328 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:55.345337 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:55.345743 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:55.346275 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:55.346290 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.347539 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.347542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:55.348713 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:55.348727 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.349549 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.349552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:55.349771 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:55.350540 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:55.350560 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:55.354088 (+ 3528us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":149,"threads_started":1}
W20260811 17:19:55.355865 24422 heartbeater.cc:644] Failed to heartbeat to 127.9.157.190:42453 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.157.190:42453: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Aug 11 17:19:55 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:19:55 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:19:55.362939 24424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:55.356428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.130:51061)
0811 17:19:55.356597 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:55.356618 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:55.356742 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:55.357185 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:55.357191 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:55.357423 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:55.357613 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:55.357623 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.358451 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.358455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:55.359414 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:55.359426 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.359983 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.359987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:55.360095 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:55.360603 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:55.360618 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:55.362762 (+ 2144us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 17:19:55.363049 24230 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:55.356485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:51061 (local address 127.9.157.190:42453)
0811 17:19:55.356606 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:55.356611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:55.356694 (+ 83us) server_negotiation.cc:407] Connection header received
0811 17:19:55.356945 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:55.356948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:55.357004 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:55.357152 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:55.357742 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.358332 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.359551 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.359848 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.359971 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:55.362913 (+ 2942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.130:51061: BlockingRecv error: recv got EOF from 127.9.157.130:51061 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 17:19:55 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:19:55 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:19:55.369980 24424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:55.363724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.130:46435)
0811 17:19:55.363862 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:55.363916 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:55.364007 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:55.364309 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:55.364312 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:55.364459 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:55.364557 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:55.364563 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.365675 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.365678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:55.366242 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:55.366251 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.366951 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.366955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:55.367131 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:55.367748 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:55.367763 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:55.369820 (+ 2057us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 17:19:55.370301 24422 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 17:19:55.370478 24230 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:55.363781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:46435 (local address 127.9.157.190:42453)
0811 17:19:55.363907 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:55.363911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:55.363985 (+ 74us) server_negotiation.cc:407] Connection header received
0811 17:19:55.364096 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:55.364099 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:55.364155 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:55.364246 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:55.364691 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.365540 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.366401 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.366814 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.366926 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:55.369969 (+ 3043us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.130:46435: BlockingRecv error: recv got EOF from 127.9.157.130:46435 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 17:19:55.471864 24425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:19:55.472204 24425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:19:55.472289 24425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:19:55.476459 24425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:19:55.476532 24425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:19:55.476642 24425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:19:55.482439 24425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:45027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/krb5kdc/oryx_127.9.157.131.keytab
--principal=oryx/127.9.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:36527
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=33929
--tserver_master_addrs=127.9.157.190:42453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:19:55.483603 24425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:19:55.484794 24425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:19:55.495384 24433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:55.495370 24436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:19:55.495370 24434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:19:55.495944 24425 server_base.cc:1034] running on GCE node
I20260811 17:19:55.496430 24425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:19:55.497107 24425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:19:55.498289 24425 hybrid_clock.cc:648] HybridClock initialized: now 1786468795498272 us; error 30 us; skew 500 ppm
Aug 11 17:19:55 dist-test-slave-mjjr krb5kdc[23164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468795, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:19:55.501487 24425 init.cc:377] Logged in from keytab as oryx/127.9.157.131@KRBTEST.COM (short username oryx)
I20260811 17:19:55.502478 24425 webserver.cc:460] Webserver started at http://127.9.157.131:33929/ using document root <none> and password file <none>
I20260811 17:19:55.503069 24425 fs_manager.cc:359] Metadata directory not provided
I20260811 17:19:55.503160 24425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:19:55.507537 24425 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:19:55.510106 24446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:55.511541 24425 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:19:55.511737 24425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "1353ceb901d040bb9881efb643f7abf5"
format_stamp: "Formatted at 2026-08-11 17:19:54 on dist-test-slave-mjjr"
server_key: "3660d0086f3bf0447ad5725f571d7e12"
server_key_iv: "f68bd3346b48ebe1854be93ee8ad6d02"
server_key_version: "encryptionkey@0"
I20260811 17:19:55.512398 24425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:19:55.530124 24425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:19:55.533969 24425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:19:55.534221 24425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:19:55.534911 24425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:19:55.535910 24425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:19:55.535983 24425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:55.536062 24425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:19:55.536113 24425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:19:55.546741 24425 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:36527
I20260811 17:19:55.546766 24603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:36527 every 8 connection(s)
I20260811 17:19:55.547916 24425 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:19:55.552680 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 24425
Aug 11 17:19:55 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:19:55 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 17:19:55.560985 24230 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:55.550232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:34713 (local address 127.9.157.190:42453)
0811 17:19:55.550447 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:55.550451 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:55.551345 (+ 894us) server_negotiation.cc:407] Connection header received
0811 17:19:55.553022 (+ 1677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:55.553026 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:55.553079 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:55.553179 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:55.554759 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.555378 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.556662 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.556882 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.556997 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:55.560853 (+ 3856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.131:34713: BlockingRecv error: recv got EOF from 127.9.157.131:34713 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 17:19:55.561703 24612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:55.550636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.131:34713)
0811 17:19:55.551158 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:55.551205 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:55.552728 (+ 1523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:55.553401 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:55.553412 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:55.553962 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:55.554600 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:55.554616 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.555518 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.555522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:55.556522 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:55.556532 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.557050 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.557054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:55.557288 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:55.558056 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:55.558074 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:55.560738 (+ 2664us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":143,"threads_started":1}
W20260811 17:19:55.562384 24607 heartbeater.cc:644] Failed to heartbeat to 127.9.157.190:42453 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.157.190:42453: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
I20260811 17:19:55.563344 24614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:55.553943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50832 (local address 127.9.157.190:42453)
0811 17:19:55.554220 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:55.554224 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:55.554236 (+ 12us) server_negotiation.cc:407] Connection header received
0811 17:19:55.554272 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:55.554274 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:55.554317 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:55.554423 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:55.555283 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.555885 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.556769 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.557031 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.557138 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:55.558727 (+ 1589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:19:55.558733 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:19:55.558750 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:19:55.558780 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:19:55.560747 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:55.561326 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:55.561333 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:55.561338 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:55.561463 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:19:55.561774 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:19:55.561777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:19:55.561779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:19:55.562310 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:19:55.562528 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 17:19:55.562890 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 17:19:55.563169 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":95,"threads_started":1}
Aug 11 17:19:55 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:19:55 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:19:55.568648 24612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:55.562930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.131:39321)
0811 17:19:55.563147 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:55.563160 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:55.563228 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:55.563502 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:55.563506 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:55.563749 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:55.563874 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:55.563882 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.564732 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.564735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:55.565349 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:55.565361 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.565939 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.565946 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:55.566042 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:55.566663 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:55.566676 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:55.568536 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141}
W20260811 17:19:55.568879 24230 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:55.563002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:39321 (local address 127.9.157.190:42453)
0811 17:19:55.563254 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:55.563259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:55.563271 (+ 12us) server_negotiation.cc:407] Connection header received
0811 17:19:55.563320 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:55.563323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:55.563371 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:55.563455 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:55.564022 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.564615 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.565474 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.565736 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.565914 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:55.568752 (+ 2838us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.131:39321: BlockingRecv error: recv got EOF from 127.9.157.131:39321 (error 108)
Metrics: {"server-negotiator.queue_time_us":177}
Aug 11 17:19:55 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:19:55 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:19:55.574297 24612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:55.569306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.131:40069)
0811 17:19:55.569425 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:55.569440 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:55.569524 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:55.569729 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:55.569731 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:55.569856 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:55.569950 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:55.569954 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.570699 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.570701 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:55.571219 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:55.571226 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.571734 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.571738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:55.571798 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:55.572391 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:55.572404 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:55.574188 (+ 1784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 17:19:55.574616 24607 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 17:19:55.574656 24230 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:55.569397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:40069 (local address 127.9.157.190:42453)
0811 17:19:55.569540 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:55.569543 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:55.569556 (+ 13us) server_negotiation.cc:407] Connection header received
0811 17:19:55.569593 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:55.569595 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:55.569630 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:55.569701 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:55.570088 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.570600 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.571335 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:55.571564 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:55.571744 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:55.574508 (+ 2764us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.131:40069: BlockingRecv error: recv got EOF from 127.9.157.131:40069 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 11 17:19:56 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:19:56 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:19:56.167714 24615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:56.161482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.129:56837)
0811 17:19:56.161951 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:56.161968 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:56.162091 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:56.162414 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:56.162420 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:56.162672 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:56.162841 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:56.162849 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:56.163768 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:56.163771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:56.164370 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:56.164378 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:56.164834 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:56.164836 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:56.164900 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:56.165454 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:56.165466 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:56.167502 (+ 2036us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":91,"threads_started":1}
W20260811 17:19:56.167834 24617 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:56.161627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:56837 (local address 127.9.157.190:42453)
0811 17:19:56.161895 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:56.161899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:56.162158 (+ 259us) server_negotiation.cc:407] Connection header received
0811 17:19:56.162211 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:56.162214 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:56.162272 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:56.162353 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:56.163020 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:56.163600 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:56.164485 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:56.164706 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:56.164786 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:56.167679 (+ 2893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.129:56837: BlockingRecv error: recv got EOF from 127.9.157.129:56837 (error 108)
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":83,"threads_started":1}
Aug 11 17:19:56 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:19:56 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:19:56.376685 24618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:56.371086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.130:60701)
0811 17:19:56.371371 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:56.371386 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:56.371491 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:56.371773 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:56.371776 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:56.371976 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:56.372080 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:56.372086 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:56.372887 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:56.372890 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:56.373465 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:56.373473 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:56.373925 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:56.373933 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:56.374034 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:56.374572 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:56.374584 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:56.376543 (+ 1959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":102,"threads_started":1}
W20260811 17:19:56.376885 24617 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:56.371122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:60701 (local address 127.9.157.190:42453)
0811 17:19:56.371277 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:56.371282 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:56.371494 (+ 212us) server_negotiation.cc:407] Connection header received
0811 17:19:56.371567 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:56.371569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:56.371621 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:56.371723 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:56.372214 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:56.372755 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:56.373574 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:56.373793 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:56.373872 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:56.376730 (+ 2858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.130:60701: BlockingRecv error: recv got EOF from 127.9.157.130:60701 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 17:19:56 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:19:56 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:19:56.581151 24619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:56.575507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.131:37981)
0811 17:19:56.575817 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:56.575836 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:56.575990 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:56.576279 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:56.576282 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:56.576504 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:56.576603 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:56.576609 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:56.577402 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:56.577407 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:56.577986 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:56.577994 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:56.578478 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:56.578482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:56.578568 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:56.579078 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:56.579090 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:56.581014 (+ 1924us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":93,"threads_started":1}
W20260811 17:19:56.581342 24617 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:56.575588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:37981 (local address 127.9.157.190:42453)
0811 17:19:56.575745 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:56.575750 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:56.575918 (+ 168us) server_negotiation.cc:407] Connection header received
0811 17:19:56.576094 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:56.576098 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:56.576155 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:56.576250 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:56.576726 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:56.577272 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:56.578113 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:56.578348 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:56.578462 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:56.581167 (+ 2705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.131:37981: BlockingRecv error: recv got EOF from 127.9.157.131:37981 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 11 17:19:57 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:19:57 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:19:57.179961 24620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:57.168764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.129:42097)
0811 17:19:57.169148 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:57.169165 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:57.169314 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:57.169727 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:57.169730 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:57.170000 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:57.170166 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:57.170176 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:57.171084 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:57.171087 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:57.171742 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:57.171752 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:57.172227 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:57.172230 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:57.172301 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:57.177631 (+ 5330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:57.177647 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:57.179794 (+ 2147us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":127,"threads_started":1}
W20260811 17:19:57.180150 24621 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:57.168957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:42097 (local address 127.9.157.190:42453)
0811 17:19:57.169256 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:57.169261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:57.169276 (+ 15us) server_negotiation.cc:407] Connection header received
0811 17:19:57.169466 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:57.169471 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:57.169539 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:57.169633 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:57.170307 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:57.170965 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:57.171877 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:57.172104 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:57.172187 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:57.179991 (+ 7804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.129:42097: BlockingRecv error: recv got EOF from 127.9.157.129:42097 (error 108)
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":110,"threads_started":1}
Aug 11 17:19:57 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:19:57 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:19:57.382928 24625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:57.377542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.130:52037)
0811 17:19:57.377842 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:57.377855 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:57.377977 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:57.378223 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:57.378225 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:57.378408 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:57.378507 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:57.378511 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:57.379264 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:57.379268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:57.379939 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:57.379950 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:57.380363 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:57.380366 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:57.380428 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:57.380927 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:57.380939 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:57.382778 (+ 1839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":107,"threads_started":1}
W20260811 17:19:57.383085 24621 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:57.377635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:52037 (local address 127.9.157.190:42453)
0811 17:19:57.377780 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:57.377784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:57.377944 (+ 160us) server_negotiation.cc:407] Connection header received
0811 17:19:57.378064 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:57.378066 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:57.378112 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:57.378185 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:57.378607 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:57.379137 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:57.380049 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:57.380263 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:57.380342 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:57.382912 (+ 2570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.130:52037: BlockingRecv error: recv got EOF from 127.9.157.130:52037 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 17:19:57 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:19:57 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:19:57.588945 24628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:57.582162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.131:39605)
0811 17:19:57.582681 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:57.582702 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:57.582890 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:57.583190 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:57.583193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:57.583425 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:57.583529 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:57.583614 (+ 85us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:57.584581 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:57.584588 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:57.585162 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:57.585171 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:57.585676 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:57.585680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:57.585773 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:57.586394 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:57.586407 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:57.588778 (+ 2371us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":130,"threads_started":1}
W20260811 17:19:57.589170 24621 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:57.582364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:39605 (local address 127.9.157.190:42453)
0811 17:19:57.582661 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:57.582674 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:57.582817 (+ 143us) server_negotiation.cc:407] Connection header received
0811 17:19:57.583004 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:57.583007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:57.583064 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:57.583157 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:57.583887 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:57.584454 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:57.585295 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:57.585544 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:57.585663 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:57.589009 (+ 3346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.131:39605: BlockingRecv error: recv got EOF from 127.9.157.131:39605 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 11 17:19:58 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:19:58 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:19:58.188651 24632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:58.181319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.129:47883)
0811 17:19:58.181687 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:58.181707 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:58.181835 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:58.182496 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:58.182500 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:58.182776 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:58.182935 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:58.182942 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:58.184030 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:58.184033 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:58.184681 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:58.184692 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:58.185373 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:58.185379 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:58.185511 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:58.186225 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:58.186240 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:58.188441 (+ 2201us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":92,"threads_started":1}
W20260811 17:19:58.188875 24633 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:58.181519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:47883 (local address 127.9.157.190:42453)
0811 17:19:58.181936 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:58.181941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:58.181968 (+ 27us) server_negotiation.cc:407] Connection header received
0811 17:19:58.182128 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:58.182132 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:58.182203 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:58.182309 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:58.183188 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:58.183854 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:58.184887 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:58.185145 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:58.185254 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:58.188684 (+ 3430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.129:47883: BlockingRecv error: recv got EOF from 127.9.157.129:47883 (error 108)
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":89,"threads_started":1}
Aug 11 17:19:58 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:19:58 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:19:58.390522 24634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:58.384204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.130:55881)
0811 17:19:58.384546 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:58.384561 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:58.384675 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:58.384998 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:58.385001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:58.385192 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:58.385297 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:58.385302 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:58.386103 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:58.386106 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:58.386699 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:58.386708 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:58.387174 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:58.387177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:58.387240 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:58.387915 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:58.387928 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:58.390392 (+ 2464us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":137,"threads_started":1}
W20260811 17:19:58.390686 24633 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:58.384334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:55881 (local address 127.9.157.190:42453)
0811 17:19:58.384503 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:58.384507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:58.384669 (+ 162us) server_negotiation.cc:407] Connection header received
0811 17:19:58.384774 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:58.384777 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:58.384825 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:58.384913 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:58.385428 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:58.385992 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:58.386823 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:58.387055 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:58.387138 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:58.390556 (+ 3418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.130:55881: BlockingRecv error: recv got EOF from 127.9.157.130:55881 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 17:19:58 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:19:58 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:19:58.596662 24635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:58.589952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.131:55605)
0811 17:19:58.590314 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:58.590331 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:58.590500 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:58.590786 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:58.590789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:58.591029 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:58.591129 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:58.591136 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:58.592098 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:58.592102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:58.592961 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:58.592971 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:58.593413 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:58.593417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:58.593515 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:58.594151 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:58.594163 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:58.596475 (+ 2312us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":125,"threads_started":1}
W20260811 17:19:58.596721 24633 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:58.590115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:55605 (local address 127.9.157.190:42453)
0811 17:19:58.590306 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:58.590311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:58.590433 (+ 122us) server_negotiation.cc:407] Connection header received
0811 17:19:58.590612 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:58.590617 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:58.590671 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:58.590762 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:58.591259 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:58.591954 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:58.593063 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:58.593278 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:58.593363 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:58.596609 (+ 3246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.131:55605: BlockingRecv error: recv got EOF from 127.9.157.131:55605 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 11 17:19:59 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:19:59 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:19:59.196532 24636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:59.189964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.129:46895)
0811 17:19:59.190475 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:59.190494 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:59.190681 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:59.191045 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:59.191049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:59.191282 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:59.191389 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:59.191421 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:59.192306 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:59.192309 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:59.192878 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:59.192886 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:59.193649 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:59.193652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:59.193723 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:59.194290 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:59.194324 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:59.196403 (+ 2079us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":117,"threads_started":1}
W20260811 17:19:59.196777 24637 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:59.190118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:46895 (local address 127.9.157.190:42453)
0811 17:19:59.190393 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:59.190398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:59.190662 (+ 264us) server_negotiation.cc:407] Connection header received
0811 17:19:59.190833 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:59.190837 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:59.190916 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:59.191019 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:59.191555 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:59.192172 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:59.193106 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:59.193423 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:59.193642 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:59.196611 (+ 2969us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.129:46895: BlockingRecv error: recv got EOF from 127.9.157.129:46895 (error 108)
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":68,"threads_started":1}
Aug 11 17:19:59 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:19:59 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:19:59.397744 24639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:59.391610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.130:42563)
0811 17:19:59.392045 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:59.392065 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:59.392217 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:59.392605 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:59.392609 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:59.392902 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:59.393038 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:59.393044 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:59.393828 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:59.393831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:59.394407 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:59.394415 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:59.394870 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:59.394872 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:59.394939 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:59.395562 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:59.395574 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:59.397608 (+ 2034us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":117,"threads_started":1}
W20260811 17:19:59.397979 24637 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:59.391810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:42563 (local address 127.9.157.190:42453)
0811 17:19:59.391999 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:59.392006 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:59.392161 (+ 155us) server_negotiation.cc:407] Connection header received
0811 17:19:59.392336 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:59.392339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:59.392435 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:59.392531 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:59.393138 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:59.393681 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:59.394530 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:59.394748 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:59.394826 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:59.397822 (+ 2996us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.130:42563: BlockingRecv error: recv got EOF from 127.9.157.130:42563 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 17:19:59 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:19:59 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:19:59.604053 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:19:59.597712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.131:55487)
0811 17:19:59.598018 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 17:19:59.598033 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:19:59.598184 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:19:59.598561 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:19:59.598563 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:19:59.598823 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:19:59.598950 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:59.598958 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:59.599990 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:59.599993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:59.600564 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:19:59.600572 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:19:59.601031 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:19:59.601034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:19:59.601101 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:59.601719 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:19:59.601733 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:19:59.603920 (+ 2187us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":99,"threads_started":1}
W20260811 17:19:59.604279 24637 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:19:59.597833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:55487 (local address 127.9.157.190:42453)
0811 17:19:59.597983 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 17:19:59.597989 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:19:59.598136 (+ 147us) server_negotiation.cc:407] Connection header received
0811 17:19:59.598351 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:19:59.598356 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:19:59.598421 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:19:59.598511 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:19:59.599084 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:59.599861 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:59.600681 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:19:59.600917 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:19:59.601005 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:19:59.604127 (+ 3122us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.131:55487: BlockingRecv error: recv got EOF from 127.9.157.131:55487 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 17:20:00 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:00 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:00.203581 24643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:00.197527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.129:44821)
0811 17:20:00.198026 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:00.198040 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:00.198172 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:00.198515 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:00.198521 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:00.198742 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:00.198876 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:00.198882 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:00.199740 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:00.199743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:00.200331 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:00.200340 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:00.200817 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:00.200820 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:00.200885 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:00.201428 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:00.201440 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:00.203444 (+ 2004us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":115,"threads_started":1}
W20260811 17:20:00.203896 24642 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:00.197569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:44821 (local address 127.9.157.190:42453)
0811 17:20:00.197877 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:00.197884 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:00.198123 (+ 239us) server_negotiation.cc:407] Connection header received
0811 17:20:00.198285 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:00.198289 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:00.198349 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:00.198458 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:00.199007 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:00.199575 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:00.200445 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:00.200675 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:00.200756 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:00.203729 (+ 2973us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.129:44821: BlockingRecv error: recv got EOF from 127.9.157.129:44821 (error 108)
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":74,"threads_started":1}
Aug 11 17:20:00 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:00 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:00.405783 24646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:00.398869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.130:45899)
0811 17:20:00.399199 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:00.399214 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:00.399312 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:00.399627 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:00.399630 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:00.399888 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:00.399992 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:00.399999 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:00.401259 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:00.401262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:00.401858 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:00.401867 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:00.402584 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:00.402587 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:00.402652 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:00.403269 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:00.403282 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:00.405638 (+ 2356us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":131,"threads_started":1}
W20260811 17:20:00.406178 24642 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:00.399041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:45899 (local address 127.9.157.190:42453)
0811 17:20:00.399216 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:00.399222 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:00.399275 (+ 53us) server_negotiation.cc:407] Connection header received
0811 17:20:00.399422 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:00.399427 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:00.399493 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:00.399597 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:00.400132 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:00.401117 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:00.401990 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:00.402448 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:00.402573 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:00.405883 (+ 3310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.130:45899: BlockingRecv error: recv got EOF from 127.9.157.130:45899 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 17:20:00 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:00 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:00.611354 24647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:00.605216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.131:52195)
0811 17:20:00.605581 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:00.605596 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:00.605722 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:00.606035 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:00.606038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:00.606244 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:00.606352 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:00.606358 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:00.607154 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:00.607157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:00.607762 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:00.607773 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:00.608212 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:00.608215 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:00.608280 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:00.608999 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:00.609022 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:00.611216 (+ 2194us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":99,"threads_started":1}
W20260811 17:20:00.611580 24642 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:00.605350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:52195 (local address 127.9.157.190:42453)
0811 17:20:00.605494 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:00.605499 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:00.605713 (+ 214us) server_negotiation.cc:407] Connection header received
0811 17:20:00.605801 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:00.605803 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:00.605853 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:00.605952 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:00.606486 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:00.607036 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:00.607888 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:00.608103 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:00.608183 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:00.611436 (+ 3253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.131:52195: BlockingRecv error: recv got EOF from 127.9.157.131:52195 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 17:20:01 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:01 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:01.210960 24649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:01.204694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.129:47899)
0811 17:20:01.205067 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:01.205083 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:01.205205 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:01.205479 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:01.205482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:01.205679 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:01.205793 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:01.205799 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:01.206601 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:01.206604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:01.207196 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:01.207205 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:01.207801 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:01.207804 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:01.207898 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:01.208515 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:01.208527 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:01.210794 (+ 2267us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":114,"threads_started":1}
W20260811 17:20:01.211158 24650 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:01.204778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:47899 (local address 127.9.157.190:42453)
0811 17:20:01.205079 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:01.205083 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:01.205176 (+ 93us) server_negotiation.cc:407] Connection header received
0811 17:20:01.205293 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:01.205296 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:01.205343 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:01.205435 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:01.205903 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:01.206479 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:01.207324 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:01.207570 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:01.207691 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:01.210996 (+ 3305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.129:47899: BlockingRecv error: recv got EOF from 127.9.157.129:47899 (error 108)
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":85,"threads_started":1}
Aug 11 17:20:01 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:01 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:01.412818 24651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:01.406681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.130:41009)
0811 17:20:01.407124 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:01.407137 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:01.407243 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:01.407477 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:01.407479 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:01.407777 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:01.407906 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:01.407913 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:01.408790 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:01.408793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:01.409391 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:01.409400 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:01.409953 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:01.409959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:01.410055 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:01.410604 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:01.410617 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:01.412680 (+ 2063us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":91,"threads_started":1}
W20260811 17:20:01.413069 24650 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:01.406820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:41009 (local address 127.9.157.190:42453)
0811 17:20:01.406995 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:01.407002 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:01.407213 (+ 211us) server_negotiation.cc:407] Connection header received
0811 17:20:01.407314 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:01.407317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:01.407364 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:01.407434 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:01.408043 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:01.408657 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:01.409533 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:01.409793 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:01.409879 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:01.412923 (+ 3044us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.130:41009: BlockingRecv error: recv got EOF from 127.9.157.130:41009 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 17:20:01 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:01 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:01.618355 24652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:01.612310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.131:38749)
0811 17:20:01.612601 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:01.612616 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:01.612752 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:01.613015 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:01.613017 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:01.613231 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:01.613334 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:01.613340 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:01.614193 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:01.614197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:01.614845 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:01.614859 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:01.615392 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:01.615395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:01.615464 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:01.616096 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:01.616109 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:01.618221 (+ 2112us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":76,"threads_started":1}
W20260811 17:20:01.618573 24650 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:01.612460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:38749 (local address 127.9.157.190:42453)
0811 17:20:01.612638 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:01.612642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:01.612674 (+ 32us) server_negotiation.cc:407] Connection header received
0811 17:20:01.612843 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:01.612846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:01.612894 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:01.612963 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:01.613475 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:01.614057 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:01.615021 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:01.615248 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:01.615341 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:01.618428 (+ 3087us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.131:38749: BlockingRecv error: recv got EOF from 127.9.157.131:38749 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 11 17:20:02 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:02 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:02.221897 24653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:02.212302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.129:60247)
0811 17:20:02.212726 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:02.212742 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:02.212851 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:02.213214 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:02.213217 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:02.213431 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:02.213560 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:02.213566 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:02.214406 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:02.214409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:02.215047 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:02.215056 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:02.215505 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:02.215507 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:02.215588 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:02.219261 (+ 3673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:02.219280 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:02.221770 (+ 2490us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":83,"threads_started":1}
W20260811 17:20:02.222093 24655 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:02.212434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:60247 (local address 127.9.157.190:42453)
0811 17:20:02.212787 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:02.212791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:02.212808 (+ 17us) server_negotiation.cc:407] Connection header received
0811 17:20:02.213022 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:02.213025 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:02.213079 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:02.213167 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:02.213724 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:02.214291 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:02.215160 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:02.215389 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:02.215474 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:02.221942 (+ 6468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.129:60247: BlockingRecv error: recv got EOF from 127.9.157.129:60247 (error 108)
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":124,"threads_started":1}
Aug 11 17:20:02 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:02 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:02.419902 24657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:02.413771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.130:39975)
0811 17:20:02.414197 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:02.414214 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:02.414389 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:02.414702 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:02.414705 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:02.414913 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:02.415014 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:02.415019 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:02.415910 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:02.415913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:02.416570 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:02.416580 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:02.417067 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:02.417070 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:02.417135 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:02.417739 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:02.417752 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:02.419777 (+ 2025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":88,"threads_started":1}
W20260811 17:20:02.420120 24655 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:02.413897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:39975 (local address 127.9.157.190:42453)
0811 17:20:02.414114 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:02.414123 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:02.414310 (+ 187us) server_negotiation.cc:407] Connection header received
0811 17:20:02.414489 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:02.414494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:02.414557 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:02.414661 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:02.415136 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:02.415782 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:02.416703 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:02.416946 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:02.417039 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:02.419966 (+ 2927us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.130:39975: BlockingRecv error: recv got EOF from 127.9.157.130:39975 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 11 17:20:02 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:02 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:02.625989 24658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:02.619693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.131:34697)
0811 17:20:02.620150 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:02.620163 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:02.620319 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:02.620625 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:02.620628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:02.620852 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:02.620956 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:02.620962 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:02.621749 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:02.621754 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:02.622322 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:02.622331 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:02.622824 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:02.622828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:02.622930 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:02.623480 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:02.623493 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:02.625801 (+ 2308us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":105,"threads_started":1}
W20260811 17:20:02.626101 24655 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:02.619758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:34697 (local address 127.9.157.190:42453)
0811 17:20:02.619968 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:02.619973 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:02.620259 (+ 286us) server_negotiation.cc:407] Connection header received
0811 17:20:02.620436 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:02.620439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:02.620499 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:02.620591 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:02.621087 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:02.621627 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:02.622459 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:02.622692 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:02.622804 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:02.625969 (+ 3165us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.131:34697: BlockingRecv error: recv got EOF from 127.9.157.131:34697 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 11 17:20:03 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:03 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:03.229207 24660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:03.222982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.129:48103)
0811 17:20:03.223321 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:03.223336 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:03.223441 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:03.223765 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:03.223768 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:03.223965 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:03.224073 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:03.224080 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:03.224920 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:03.224923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:03.225523 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:03.225533 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:03.226054 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:03.226058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:03.226142 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:03.226687 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:03.226700 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:03.228999 (+ 2299us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":125,"threads_started":1}
W20260811 17:20:03.229347 24661 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:03.223179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:48103 (local address 127.9.157.190:42453)
0811 17:20:03.223434 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:03.223438 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:03.223454 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:20:03.223526 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:03.223529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:03.223573 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:03.223709 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:03.224210 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:03.224764 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:03.225652 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:03.225903 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:03.226003 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:03.229173 (+ 3170us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.129:48103: BlockingRecv error: recv got EOF from 127.9.157.129:48103 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":91,"threads_started":1}
Aug 11 17:20:03 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:03 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:03.427333 24662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:03.420994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.130:47441)
0811 17:20:03.421407 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:03.421426 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:03.421599 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:03.421920 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:03.421923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:03.422189 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:03.422319 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:03.422326 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:03.423363 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:03.423368 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:03.423987 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:03.423996 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:03.424474 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:03.424477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:03.424563 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:03.425120 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:03.425132 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:03.427147 (+ 2015us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":126,"threads_started":1}
W20260811 17:20:03.427443 24661 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:03.421181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:47441 (local address 127.9.157.190:42453)
0811 17:20:03.421409 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:03.421416 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:03.421528 (+ 112us) server_negotiation.cc:407] Connection header received
0811 17:20:03.421714 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:03.421717 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:03.421770 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:03.421865 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:03.422451 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:03.423237 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:03.424118 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:03.424352 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:03.424466 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:03.427320 (+ 2854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.130:47441: BlockingRecv error: recv got EOF from 127.9.157.130:47441 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Aug 11 17:20:03 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:03 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:03.633545 24665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:03.627190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.131:50705)
0811 17:20:03.627524 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:03.627540 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:03.627717 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:03.628110 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:03.628113 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:03.628359 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:03.628503 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:03.628511 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:03.629499 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:03.629502 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:03.630081 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:03.630090 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:03.630602 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:03.630605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:03.630676 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:03.631282 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:03.631295 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:03.633417 (+ 2122us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":130,"threads_started":1}
W20260811 17:20:03.633777 24661 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:03.627326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:50705 (local address 127.9.157.190:42453)
0811 17:20:03.627523 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:03.627529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:03.627678 (+ 149us) server_negotiation.cc:407] Connection header received
0811 17:20:03.627864 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:03.627866 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:03.627952 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:03.628053 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:03.628664 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:03.629379 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:03.630212 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:03.630458 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:03.630549 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:03.633625 (+ 3076us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.131:50705: BlockingRecv error: recv got EOF from 127.9.157.131:50705 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 11 17:20:04 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:04 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:04.236627 24666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:04.230226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.129:38249)
0811 17:20:04.230548 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:04.230568 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:04.230685 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:04.231011 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:04.231014 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:04.231221 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:04.231331 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:04.231336 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:04.232209 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:04.232212 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:04.232784 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:04.232793 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:04.233255 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:04.233257 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:04.233318 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:04.233933 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:04.233945 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:04.236481 (+ 2536us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":86,"threads_started":1}
W20260811 17:20:04.236788 24667 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:04.230409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:38249 (local address 127.9.157.190:42453)
0811 17:20:04.230721 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:04.230724 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:04.230740 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:20:04.230789 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:04.230792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:04.230835 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:04.230929 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:04.231499 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:04.232092 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:04.232909 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:04.233142 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:04.233235 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:04.236638 (+ 3403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.129:38249: BlockingRecv error: recv got EOF from 127.9.157.129:38249 (error 108)
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":88,"threads_started":1}
Aug 11 17:20:04 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:04 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:04.434486 24668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:04.428403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.130:37837)
0811 17:20:04.428804 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:04.428819 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:04.428973 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:04.429288 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:04.429291 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:04.429542 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:04.429668 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:04.429675 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:04.430503 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:04.430506 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:04.431078 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:04.431086 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:04.431572 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:04.431576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:04.431698 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:04.432300 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:04.432312 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:04.434270 (+ 1958us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":98,"threads_started":1}
W20260811 17:20:04.434543 24667 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:04.428499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:37837 (local address 127.9.157.190:42453)
0811 17:20:04.428661 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:04.428665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:04.428924 (+ 259us) server_negotiation.cc:407] Connection header received
0811 17:20:04.429098 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:04.429104 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:04.429159 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:04.429253 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:04.429793 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:04.430389 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:04.431212 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:04.431445 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:04.431555 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:04.434427 (+ 2872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.130:37837: BlockingRecv error: recv got EOF from 127.9.157.130:37837 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 17:20:04 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:04 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:04.640712 24672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:04.634536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.131:55689)
0811 17:20:04.634830 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:04.634847 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:04.634967 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:04.635296 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:04.635299 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:04.635507 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:04.635606 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:04.635611 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:04.636679 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:04.636684 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:04.637239 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:04.637246 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:04.637736 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:04.637740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:04.637829 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:04.638375 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:04.638387 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:04.640497 (+ 2110us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":104,"threads_started":1}
W20260811 17:20:04.640790 24667 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:04.634711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:55689 (local address 127.9.157.190:42453)
0811 17:20:04.634881 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:04.634889 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:04.634910 (+ 21us) server_negotiation.cc:407] Connection header received
0811 17:20:04.635100 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:04.635106 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:04.635175 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:04.635264 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:04.635926 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:04.636550 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:04.637374 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:04.637602 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:04.637717 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:04.640656 (+ 2939us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.131:55689: BlockingRecv error: recv got EOF from 127.9.157.131:55689 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 17:20:05 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:05 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:05.244045 24673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:05.237757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.129:35031)
0811 17:20:05.238147 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:05.238162 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:05.238259 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:05.238515 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:05.238518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:05.238710 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:05.238836 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:05.238841 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:05.239809 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:05.239812 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:05.240403 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:05.240412 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:05.240862 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:05.240865 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:05.240943 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:05.241562 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:05.241575 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:05.243691 (+ 2116us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":97,"threads_started":1}
W20260811 17:20:05.244174 24674 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:05.237953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:35031 (local address 127.9.157.190:42453)
0811 17:20:05.238211 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:05.238215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:05.238231 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:20:05.238338 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:05.238341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:05.238383 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:05.238463 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:05.238986 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:05.239561 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:05.240522 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:05.240748 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:05.240829 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:05.243983 (+ 3154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.129:35031: BlockingRecv error: recv got EOF from 127.9.157.129:35031 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":65,"threads_started":1}
Aug 11 17:20:05 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:05 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:05.441920 24677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:05.435632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.130:44387)
0811 17:20:05.435990 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:05.436005 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:05.436115 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:05.436394 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:05.436396 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:05.436607 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:05.436713 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:05.436725 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:05.437715 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:05.437719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:05.438364 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:05.438375 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:05.438901 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:05.438905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:05.438983 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:05.439617 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:05.439630 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:05.441792 (+ 2162us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":124,"threads_started":1}
W20260811 17:20:05.442124 24674 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:05.435807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:44387 (local address 127.9.157.190:42453)
0811 17:20:05.436001 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:05.436007 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:05.436110 (+ 103us) server_negotiation.cc:407] Connection header received
0811 17:20:05.436214 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:05.436217 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:05.436267 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:05.436353 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:05.437016 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:05.437569 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:05.438488 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:05.438724 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:05.438904 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:05.441972 (+ 3068us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.130:44387: BlockingRecv error: recv got EOF from 127.9.157.130:44387 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 11 17:20:05 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:05 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:05.647269 24678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:05.641624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.131:42107)
0811 17:20:05.641950 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:05.641964 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:05.642063 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:05.642311 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:05.642313 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:05.642499 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:05.642599 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:05.642604 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:05.643400 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:05.643403 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:05.644045 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:05.644056 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:05.644548 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:05.644551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:05.644625 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:05.645249 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:05.645261 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:05.647139 (+ 1878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":111,"threads_started":1}
W20260811 17:20:05.647501 24674 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:05.641794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:42107 (local address 127.9.157.190:42453)
0811 17:20:05.641956 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:05.641966 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:05.642067 (+ 101us) server_negotiation.cc:407] Connection header received
0811 17:20:05.642137 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:05.642140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:05.642190 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:05.642267 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:05.642710 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:05.643245 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:05.644172 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:05.644402 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:05.644495 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:05.647350 (+ 2855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.131:42107: BlockingRecv error: recv got EOF from 127.9.157.131:42107 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 11 17:20:06 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:06 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:06.251343 24679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:06.245062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.129:37979)
0811 17:20:06.245408 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:06.245428 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:06.245564 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:06.245901 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:06.245904 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:06.246184 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:06.246339 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:06.246347 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:06.247268 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:06.247271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:06.247950 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:06.247959 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:06.248444 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:06.248447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:06.248518 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:06.249044 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:06.249058 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:06.251177 (+ 2119us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":97,"threads_started":1}
W20260811 17:20:06.251781 24680 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:06.245266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:37979 (local address 127.9.157.190:42453)
0811 17:20:06.245532 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:06.245536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:06.245550 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:20:06.245698 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:06.245701 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:06.245755 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:06.245859 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:06.246506 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:06.247154 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:06.248071 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:06.248301 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:06.248396 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:06.251387 (+ 2991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.129:37979: BlockingRecv error: recv got EOF from 127.9.157.129:37979 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":76,"threads_started":1}
Aug 11 17:20:06 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:06 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:06.448817 24681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:06.442748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.130:36411)
0811 17:20:06.443131 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:06.443145 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:06.443290 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:06.443585 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:06.443588 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:06.443857 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:06.443959 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:06.443965 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:06.444747 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:06.444751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:06.445323 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:06.445332 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:06.445812 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:06.445815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:06.445903 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:06.446455 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:06.446466 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:06.448561 (+ 2095us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":125,"threads_started":1}
W20260811 17:20:06.448957 24680 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:06.442868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:36411 (local address 127.9.157.190:42453)
0811 17:20:06.443065 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:06.443070 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:06.443228 (+ 158us) server_negotiation.cc:407] Connection header received
0811 17:20:06.443404 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:06.443407 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:06.443461 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:06.443554 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:06.444094 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:06.444627 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:06.445450 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:06.445676 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:06.445787 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:06.448821 (+ 3034us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.130:36411: BlockingRecv error: recv got EOF from 127.9.157.130:36411 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 11 17:20:06 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:06 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:06.653788 24683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:06.648138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.131:45319)
0811 17:20:06.648482 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:06.648495 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:06.648593 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:06.648844 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:06.648846 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:06.649046 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:06.649144 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:06.649157 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:06.649899 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:06.649902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:06.650476 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:06.650485 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:06.650947 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:06.650950 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:06.651016 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:06.651602 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:06.651615 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:06.653660 (+ 2045us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":117,"threads_started":1}
W20260811 17:20:06.653982 24680 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:06.648275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:45319 (local address 127.9.157.190:42453)
0811 17:20:06.648433 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:06.648437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:06.648563 (+ 126us) server_negotiation.cc:407] Connection header received
0811 17:20:06.648673 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:06.648675 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:06.648724 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:06.648802 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:06.649262 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:06.649791 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:06.650590 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:06.650831 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:06.650915 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:06.653832 (+ 2917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.131:45319: BlockingRecv error: recv got EOF from 127.9.157.131:45319 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 17:20:07 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:07 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:07.263535 24685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:07.252426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.129:36795)
0811 17:20:07.252846 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:07.252862 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:07.252976 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:07.253283 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:07.253287 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:07.253538 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:07.253676 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:07.253684 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:07.254501 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:07.254505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:07.255181 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:07.255193 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:07.255736 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:07.255739 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:07.255831 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:07.261263 (+ 5432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:07.261281 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:07.263350 (+ 2069us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":92,"threads_started":1}
W20260811 17:20:07.263729 24686 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:07.252506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:36795 (local address 127.9.157.190:42453)
0811 17:20:07.252817 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:07.252823 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:07.252951 (+ 128us) server_negotiation.cc:407] Connection header received
0811 17:20:07.253078 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:07.253081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:07.253146 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:07.253258 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:07.253805 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:07.254378 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:07.255309 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:07.255580 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:07.255728 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:07.263550 (+ 7822us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.129:36795: BlockingRecv error: recv got EOF from 127.9.157.129:36795 (error 108)
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":107,"threads_started":1}
Aug 11 17:20:07 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:07 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:07.456027 24687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:07.449761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.130:54055)
0811 17:20:07.450168 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:07.450181 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:07.450325 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:07.450621 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:07.450623 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:07.450846 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:07.450948 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:07.450954 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:07.451826 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:07.451831 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:07.452398 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:07.452406 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:07.452895 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:07.452899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:07.452987 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:07.453599 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:07.453612 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:07.455845 (+ 2233us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":99,"threads_started":1}
W20260811 17:20:07.456128 24686 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:07.449818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:54055 (local address 127.9.157.190:42453)
0811 17:20:07.450096 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:07.450102 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:07.450258 (+ 156us) server_negotiation.cc:407] Connection header received
0811 17:20:07.450449 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:07.450454 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:07.450505 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:07.450598 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:07.451067 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:07.451606 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:07.452537 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:07.452770 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:07.452878 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:07.456002 (+ 3124us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.130:54055: BlockingRecv error: recv got EOF from 127.9.157.130:54055 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 17:20:07 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:07 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:07.660805 24688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:07.654686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.131:39799)
0811 17:20:07.655013 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:07.655065 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:07.655210 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:07.655533 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:07.655536 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:07.655794 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:07.655898 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:07.655905 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:07.656782 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:07.656786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:07.657430 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:07.657440 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:07.657974 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:07.657978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:07.658060 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:07.658650 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:07.658672 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:07.660620 (+ 1948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":119,"threads_started":1}
W20260811 17:20:07.660880 24686 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:07.654799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:39799 (local address 127.9.157.190:42453)
0811 17:20:07.654936 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:07.654941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:07.655171 (+ 230us) server_negotiation.cc:407] Connection header received
0811 17:20:07.655347 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:07.655351 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:07.655409 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:07.655502 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:07.656021 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:07.656678 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:07.657562 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:07.657848 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:07.657967 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:07.660759 (+ 2792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.131:39799: BlockingRecv error: recv got EOF from 127.9.157.131:39799 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 11 17:20:08 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:08 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:08.270571 24691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:08.264643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.129:48745)
0811 17:20:08.265020 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:08.265036 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:08.265140 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:08.265462 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:08.265465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:08.265673 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:08.265787 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:08.265792 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:08.266644 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:08.266647 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:08.267238 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:08.267248 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:08.267812 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:08.267815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:08.267899 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:08.268428 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:08.268440 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:08.270370 (+ 1930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":101,"threads_started":1}
W20260811 17:20:08.270855 24692 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:08.264807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:48745 (local address 127.9.157.190:42453)
0811 17:20:08.265073 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:08.265077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:08.265092 (+ 15us) server_negotiation.cc:407] Connection header received
0811 17:20:08.265247 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:08.265250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:08.265299 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:08.265385 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:08.265938 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:08.266509 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:08.267356 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:08.267599 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:08.267789 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:08.270574 (+ 2785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.129:48745: BlockingRecv error: recv got EOF from 127.9.157.129:48745 (error 108)
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":67,"threads_started":1}
Aug 11 17:20:08 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:08 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:08.462982 24695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:08.457023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.130:46327)
0811 17:20:08.457340 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:08.457354 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:08.457467 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:08.457723 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:08.457725 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:08.457943 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:08.458085 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:08.458092 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:08.458920 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:08.458923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:08.459486 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:08.459494 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:08.460041 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:08.460044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:08.460106 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:08.460691 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:08.460703 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:08.462861 (+ 2158us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":100,"threads_started":1}
W20260811 17:20:08.463167 24692 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:08.457139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:46327 (local address 127.9.157.190:42453)
0811 17:20:08.457351 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:08.457375 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:08.457423 (+ 48us) server_negotiation.cc:407] Connection header received
0811 17:20:08.457559 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:08.457561 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:08.457611 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:08.457691 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:08.458203 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:08.458810 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:08.459599 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:08.459920 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:08.460010 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:08.463019 (+ 3009us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.130:46327: BlockingRecv error: recv got EOF from 127.9.157.130:46327 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 11 17:20:08 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:08 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:08.667100 24696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:08.661676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.131:46481)
0811 17:20:08.661976 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:08.661991 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:08.662101 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:08.662371 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:08.662373 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:08.662580 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:08.662684 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:08.662690 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:08.663430 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:08.663434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:08.664034 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:08.664044 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:08.664461 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:08.664464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:08.664536 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:08.665034 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:08.665046 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:08.666972 (+ 1926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":107,"threads_started":1}
W20260811 17:20:08.667272 24692 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:08.661766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:46481 (local address 127.9.157.190:42453)
0811 17:20:08.661921 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:08.661925 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:08.662063 (+ 138us) server_negotiation.cc:407] Connection header received
0811 17:20:08.662203 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:08.662205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:08.662253 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:08.662325 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:08.662788 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:08.663327 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:08.664137 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:08.664352 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:08.664431 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:08.667130 (+ 2699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.131:46481: BlockingRecv error: recv got EOF from 127.9.157.131:46481 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 17:20:09 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:09 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:09.278395 24699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:09.271857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.129:55141)
0811 17:20:09.272230 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:09.272249 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:09.272378 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:09.272988 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:09.272991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:09.273251 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:09.273383 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:09.273390 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:09.274236 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:09.274241 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:09.274847 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:09.274856 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:09.275394 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:09.275397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:09.275492 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:09.276105 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:09.276119 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:09.278231 (+ 2112us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":127,"threads_started":1}
W20260811 17:20:09.278595 24703 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:09.272376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:55141 (local address 127.9.157.190:42453)
0811 17:20:09.272734 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:09.272738 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:09.272751 (+ 13us) server_negotiation.cc:407] Connection header received
0811 17:20:09.272787 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:09.272789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:09.272831 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:09.272951 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:09.273516 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:09.274102 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:09.275009 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:09.275253 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:09.275378 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:09.278405 (+ 3027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.129:55141: BlockingRecv error: recv got EOF from 127.9.157.129:55141 (error 108)
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":97,"threads_started":1}
Aug 11 17:20:09 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:09 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:09.469970 24707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:09.464061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.130:49677)
0811 17:20:09.464391 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:09.464409 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:09.464528 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:09.464894 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:09.464897 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:09.465114 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:09.465264 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:09.465274 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:09.466172 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:09.466175 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:09.466798 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:09.466807 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:09.467241 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:09.467243 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:09.467309 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:09.467930 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:09.467942 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:09.469836 (+ 1894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":79,"threads_started":1}
W20260811 17:20:09.470165 24703 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:09.464160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:49677 (local address 127.9.157.190:42453)
0811 17:20:09.464336 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:09.464348 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:09.464490 (+ 142us) server_negotiation.cc:407] Connection header received
0811 17:20:09.464630 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:09.464633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:09.464694 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:09.464823 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:09.465379 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:09.466058 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:09.466915 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:09.467134 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:09.467215 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:09.469998 (+ 2783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.130:49677: BlockingRecv error: recv got EOF from 127.9.157.130:49677 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 17:20:09 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:09 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.131@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:09.674086 24708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:09.668201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.131:52829)
0811 17:20:09.668477 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:09.668495 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:09.668620 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:09.668898 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:09.668901 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:09.669092 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:09.669213 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:09.669219 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:09.670101 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:09.670106 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:09.670751 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:09.670762 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:09.671260 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:09.671264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:09.671371 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:09.671968 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:09.671980 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:09.673940 (+ 1960us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":100,"threads_started":1}
W20260811 17:20:09.674238 24703 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:09.668281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:52829 (local address 127.9.157.190:42453)
0811 17:20:09.668438 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:09.668442 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:09.668584 (+ 142us) server_negotiation.cc:407] Connection header received
0811 17:20:09.668707 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:09.668710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:09.668758 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:09.668843 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:09.669350 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:09.669980 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:09.670876 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:09.671131 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:09.671242 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:09.674084 (+ 2842us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.131:52829: BlockingRecv error: recv got EOF from 127.9.157.131:52829 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 17:20:10 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:10 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.129@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:10.285074 24710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:10.279489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.129:40733)
0811 17:20:10.279832 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:10.279853 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:10.279993 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:10.280307 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:10.280310 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:10.280504 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:10.280602 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:10.280608 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:10.281366 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:10.281369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:10.281929 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:10.281937 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:10.282354 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:10.282357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:10.282417 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:10.283020 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:10.283031 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:10.284949 (+ 1918us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":78,"threads_started":1}
W20260811 17:20:10.285285 24714 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:10.279709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:40733 (local address 127.9.157.190:42453)
0811 17:20:10.280022 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:10.280026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:10.280055 (+ 29us) server_negotiation.cc:407] Connection header received
0811 17:20:10.280098 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:10.280100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:10.280143 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:10.280217 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:10.280719 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:10.281264 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:10.282034 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:10.282254 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:10.282330 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:10.285103 (+ 2773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.129:40733: BlockingRecv error: recv got EOF from 127.9.157.129:40733 (error 108)
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":114,"threads_started":1}
Aug 11 17:20:10 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 17:20:10 dist-test-slave-mjjr krb5kdc[23164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.157.130@KRBTEST.COM for oryx/127.9.157.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 17:20:10.477317 24716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:10.470937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42453 (local address 127.9.157.130:43343)
0811 17:20:10.471327 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:10.471341 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:10.471494 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:10.471882 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:10.471885 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:10.472118 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:10.472226 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:10.472232 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:10.473100 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:10.473106 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:10.473732 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:10.473741 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:10.474269 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:10.474273 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:10.474376 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:10.474979 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:10.474992 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:10.477142 (+ 2150us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.157.190:42453: Server oryx/127.9.157.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":101,"threads_started":1}
W20260811 17:20:10.477361 24714 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:10.471036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:43343 (local address 127.9.157.190:42453)
0811 17:20:10.471220 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:10.471225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:10.471428 (+ 203us) server_negotiation.cc:407] Connection header received
0811 17:20:10.471610 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:10.471614 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:10.471747 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:10.471846 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:10.472363 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:10.472972 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:10.473868 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:10.474123 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:10.474245 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:10.477261 (+ 3016us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.157.130:43343: BlockingRecv error: recv got EOF from 127.9.157.130:43343 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 17:20:10.560568 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 24018
I20260811 17:20:10.572743 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 24233
I20260811 17:20:10.588313 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 24425
I20260811 17:20:10.608453 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 23916
2026-08-11T17:20:10Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18531 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T17:20:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:20:10Z Disabled control of system clock
I20260811 17:20:10.651126 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:40799
--webserver_interface=127.9.157.190
--webserver_port=0
--builtin_ntp_servers=127.9.157.148:35917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:40799
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 17:20:10.771454 24722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:20:10.771888 24722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:20:10.771932 24722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:20:10.776072 24722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:20:10.776141 24722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:20:10.776157 24722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:20:10.776175 24722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:20:10.776189 24722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:20:10.781306 24722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:35917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:40799
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:40799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:20:10.782517 24722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:20:10.783955 24722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:20:10.792933 24737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:20:10.792927 24732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:20:10.792946 24730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:20:10.793412 24722 server_base.cc:1034] running on GCE node
I20260811 17:20:10.794070 24722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:20:10.795548 24722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:20:10.796753 24722 hybrid_clock.cc:648] HybridClock initialized: now 1786468810796732 us; error 35 us; skew 500 ppm
I20260811 17:20:10.798870 24722 webserver.cc:460] Webserver started at http://127.9.157.190:33589/ using document root <none> and password file <none>
I20260811 17:20:10.799522 24722 fs_manager.cc:359] Metadata directory not provided
I20260811 17:20:10.799624 24722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:20:10.799947 24722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:20:10.802241 24722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "57cde687eaf54d4ea732d8163c101609"
format_stamp: "Formatted at 2026-08-11 17:20:10 on dist-test-slave-mjjr"
server_key: "c51e03a593f50d986f29586ae06fdecc"
server_key_iv: "51631f56fe720d979469596aef5140a9"
server_key_version: "encryptionkey@0"
I20260811 17:20:10.802901 24722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "57cde687eaf54d4ea732d8163c101609"
format_stamp: "Formatted at 2026-08-11 17:20:10 on dist-test-slave-mjjr"
server_key: "c51e03a593f50d986f29586ae06fdecc"
server_key_iv: "51631f56fe720d979469596aef5140a9"
server_key_version: "encryptionkey@0"
I20260811 17:20:10.807047 24722 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:20:10.809969 24747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:10.811301 24722 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 17:20:10.811479 24722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "57cde687eaf54d4ea732d8163c101609"
format_stamp: "Formatted at 2026-08-11 17:20:10 on dist-test-slave-mjjr"
server_key: "c51e03a593f50d986f29586ae06fdecc"
server_key_iv: "51631f56fe720d979469596aef5140a9"
server_key_version: "encryptionkey@0"
I20260811 17:20:10.811630 24722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:20:10.827688 24722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:20:10.828799 24722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:20:10.829113 24722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:20:10.837323 24722 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:40799
I20260811 17:20:10.837325 24824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:40799 every 8 connection(s)
I20260811 17:20:10.838488 24722 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:20:10.841706 24825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:20:10.847791 24825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57cde687eaf54d4ea732d8163c101609: Bootstrap starting.
I20260811 17:20:10.848423 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 24722
I20260811 17:20:10.848574 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:20:10.848860 9846 external_mini_cluster.cc:1428] Setting key ef34298fb9df27b245037240ca45f4e6
I20260811 17:20:10.850896 24825 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57cde687eaf54d4ea732d8163c101609: Neither blocks nor log segments found. Creating new log.
I20260811 17:20:10.851758 24825 log.cc:824] T 00000000000000000000000000000000 P 57cde687eaf54d4ea732d8163c101609: Log is configured to *not* fsync() on all Append() calls
I20260811 17:20:10.853814 24825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57cde687eaf54d4ea732d8163c101609: No bootstrap required, opened a new log
I20260811 17:20:10.857580 24825 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57cde687eaf54d4ea732d8163c101609 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57cde687eaf54d4ea732d8163c101609" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 40799 } }
I20260811 17:20:10.857893 24825 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57cde687eaf54d4ea732d8163c101609 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:20:10.858001 24825 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57cde687eaf54d4ea732d8163c101609 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57cde687eaf54d4ea732d8163c101609, State: Initialized, Role: FOLLOWER
I20260811 17:20:10.858508 24825 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57cde687eaf54d4ea732d8163c101609 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57cde687eaf54d4ea732d8163c101609" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 40799 } }
I20260811 17:20:10.858669 24825 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57cde687eaf54d4ea732d8163c101609 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:20:10.858760 24825 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57cde687eaf54d4ea732d8163c101609 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:20:10.858883 24825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57cde687eaf54d4ea732d8163c101609 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:20:10.858948 24831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:10.850281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39422 (local address 127.9.157.190:40799)
0811 17:20:10.851027 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:10.851052 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:10.851102 (+ 50us) server_negotiation.cc:407] Connection header received
0811 17:20:10.851859 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:10.851907 (+ 48us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:10.852350 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:10.852709 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:10.853170 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:10.854235 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:10.855157 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:10.855733 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:10.855922 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:10.856840 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:20:10.856846 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:20:10.856861 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:20:10.857294 (+ 433us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:20:10.857356 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:20:10.857380 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:20:10.857496 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 17:20:10.857645 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 17:20:10.857983 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":142,"threads_started":1}
I20260811 17:20:10.860211 24825 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57cde687eaf54d4ea732d8163c101609 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57cde687eaf54d4ea732d8163c101609" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 40799 } }
I20260811 17:20:10.860586 24825 leader_election.cc:302] T 00000000000000000000000000000000 P 57cde687eaf54d4ea732d8163c101609 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57cde687eaf54d4ea732d8163c101609; no voters:
I20260811 17:20:10.860962 24825 leader_election.cc:288] T 00000000000000000000000000000000 P 57cde687eaf54d4ea732d8163c101609 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:20:10.861047 24833 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57cde687eaf54d4ea732d8163c101609 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:20:10.861282 24833 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57cde687eaf54d4ea732d8163c101609 [term 1 LEADER]: Becoming Leader. State: Replica: 57cde687eaf54d4ea732d8163c101609, State: Running, Role: LEADER
I20260811 17:20:10.861991 24833 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57cde687eaf54d4ea732d8163c101609 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57cde687eaf54d4ea732d8163c101609" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 40799 } }
I20260811 17:20:10.862396 24825 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57cde687eaf54d4ea732d8163c101609 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:20:10.863298 24833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57cde687eaf54d4ea732d8163c101609 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57cde687eaf54d4ea732d8163c101609. Latest consensus state: current_term: 1 leader_uuid: "57cde687eaf54d4ea732d8163c101609" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57cde687eaf54d4ea732d8163c101609" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 40799 } } }
I20260811 17:20:10.863533 24833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57cde687eaf54d4ea732d8163c101609 [sys.catalog]: This master's current role is: LEADER
I20260811 17:20:10.863791 24835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57cde687eaf54d4ea732d8163c101609 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57cde687eaf54d4ea732d8163c101609" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57cde687eaf54d4ea732d8163c101609" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 40799 } } }
I20260811 17:20:10.863987 24835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57cde687eaf54d4ea732d8163c101609 [sys.catalog]: This master's current role is: LEADER
W20260811 17:20:10.867509 24856 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 57cde687eaf54d4ea732d8163c101609: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:20:10.867587 24856 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:20:10.867688 24848 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:20:10.868400 24848 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:20:10.873764 24848 catalog_manager.cc:1347] Generated new cluster ID: 3ad27c76628c4f90ad823b3a4c482e12
I20260811 17:20:10.873847 24848 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:20:10.901419 24848 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:20:10.902518 24848 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:20:10.914229 24848 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57cde687eaf54d4ea732d8163c101609: Generated new TSK 0
I20260811 17:20:10.915266 24848 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:20:10.919788 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:0
--local_ip_for_outbound_sockets=127.9.157.129
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:40799
--builtin_ntp_servers=127.9.157.148:35917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 17:20:11.041227 24860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:20:11.041620 24860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:20:11.041709 24860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:20:11.046044 24860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:20:11.046146 24860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:20:11.046244 24860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:20:11.051246 24860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:35917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:40799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:20:11.052635 24860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:20:11.054121 24860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:20:11.064648 24873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:20:11.064671 24869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:20:11.065006 24860 server_base.cc:1034] running on GCE node
W20260811 17:20:11.064741 24868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:20:11.065448 24860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:20:11.066154 24860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:20:11.067354 24860 hybrid_clock.cc:648] HybridClock initialized: now 1786468811067332 us; error 31 us; skew 500 ppm
I20260811 17:20:11.069329 24860 webserver.cc:460] Webserver started at http://127.9.157.129:39065/ using document root <none> and password file <none>
I20260811 17:20:11.069965 24860 fs_manager.cc:359] Metadata directory not provided
I20260811 17:20:11.070066 24860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:20:11.070319 24860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:20:11.072594 24860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/data/instance:
uuid: "094e01df97604061bab3385e50b2503d"
format_stamp: "Formatted at 2026-08-11 17:20:11 on dist-test-slave-mjjr"
server_key: "d930d72ccf0f521348f99ba5403a992b"
server_key_iv: "c50b56889d19b03f7914f70f5c1b53d9"
server_key_version: "encryptionkey@0"
I20260811 17:20:11.073199 24860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance:
uuid: "094e01df97604061bab3385e50b2503d"
format_stamp: "Formatted at 2026-08-11 17:20:11 on dist-test-slave-mjjr"
server_key: "d930d72ccf0f521348f99ba5403a992b"
server_key_iv: "c50b56889d19b03f7914f70f5c1b53d9"
server_key_version: "encryptionkey@0"
I20260811 17:20:11.077198 24860 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:20:11.080065 24885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:11.081418 24860 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:20:11.081573 24860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "094e01df97604061bab3385e50b2503d"
format_stamp: "Formatted at 2026-08-11 17:20:11 on dist-test-slave-mjjr"
server_key: "d930d72ccf0f521348f99ba5403a992b"
server_key_iv: "c50b56889d19b03f7914f70f5c1b53d9"
server_key_version: "encryptionkey@0"
I20260811 17:20:11.081709 24860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:20:11.096261 24860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:20:11.097093 24860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:20:11.097327 24860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:20:11.098001 24860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:20:11.099051 24860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:20:11.099131 24860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:11.099200 24860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:20:11.099251 24860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:11.109588 24860 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:46493
I20260811 17:20:11.109623 25040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:46493 every 8 connection(s)
I20260811 17:20:11.110638 24860 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:20:11.118587 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 24860
I20260811 17:20:11.118752 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance
I20260811 17:20:11.119066 9846 external_mini_cluster.cc:1428] Setting key f31afd06e525783962d3b18f6a10b301
I20260811 17:20:11.121438 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:0
--local_ip_for_outbound_sockets=127.9.157.130
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:40799
--builtin_ntp_servers=127.9.157.148:35917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 17:20:11.121686 24831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:11.112775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:60381 (local address 127.9.157.190:40799)
0811 17:20:11.112940 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:11.112945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:11.114192 (+ 1247us) server_negotiation.cc:407] Connection header received
0811 17:20:11.115778 (+ 1586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:11.115781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:11.115832 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:11.115913 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:11.117027 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:11.117588 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:11.119210 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:11.119523 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:11.119619 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:11.121090 (+ 1471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:20:11.121096 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:20:11.121103 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:20:11.121119 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:20:11.121145 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:20:11.121158 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:20:11.121247 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 17:20:11.121419 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 17:20:11.121490 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 17:20:11.122658 25044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:11.113276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:40799 (local address 127.9.157.129:60381)
0811 17:20:11.114020 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:11.114068 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:11.115481 (+ 1413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:11.116106 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:11.116113 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:11.116204 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:11.116862 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:11.116885 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:11.117744 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:11.117748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:11.119028 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:11.119040 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:11.119769 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:11.119775 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:11.120051 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:11.120798 (+ 747us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:20:11.120799 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:11.120846 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:20:11.120924 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:20:11.121254 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:20:11.121263 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:20:11.121267 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 17:20:11.121375 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 17:20:11.121734 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":126,"threads_started":1}
I20260811 17:20:11.124091 25041 heartbeater.cc:347] Connected to a master server at 127.9.157.190:40799
I20260811 17:20:11.124354 25041 heartbeater.cc:464] Registering TS with master...
I20260811 17:20:11.124953 25041 heartbeater.cc:511] Master 127.9.157.190:40799 requested a full tablet report, sending...
I20260811 17:20:11.126797 24775 ts_manager.cc:194] Registered new tserver with Master: 094e01df97604061bab3385e50b2503d (127.9.157.129:46493)
I20260811 17:20:11.128674 24775 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.157.129:60381
W20260811 17:20:11.243777 25046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:20:11.244083 25046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:20:11.244123 25046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:20:11.248263 25046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:20:11.248335 25046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:20:11.248392 25046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:20:11.253234 25046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:35917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:40799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:20:11.254406 25046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:20:11.255874 25046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:20:11.266508 25052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:20:11.266508 25051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:20:11.266512 25055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:20:11.266803 25046 server_base.cc:1034] running on GCE node
I20260811 17:20:11.267328 25046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:20:11.268071 25046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:20:11.269269 25046 hybrid_clock.cc:648] HybridClock initialized: now 1786468811269238 us; error 48 us; skew 500 ppm
I20260811 17:20:11.271483 25046 webserver.cc:460] Webserver started at http://127.9.157.130:41111/ using document root <none> and password file <none>
I20260811 17:20:11.272207 25046 fs_manager.cc:359] Metadata directory not provided
I20260811 17:20:11.272326 25046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:20:11.272584 25046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:20:11.274909 25046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/data/instance:
uuid: "3faa090f008a40789e985cc6f61fb53d"
format_stamp: "Formatted at 2026-08-11 17:20:11 on dist-test-slave-mjjr"
server_key: "5c71bcf76e69a3bd8369476bc54cc684"
server_key_iv: "2fb6813fa432aa8eddea12e88e4d4a94"
server_key_version: "encryptionkey@0"
I20260811 17:20:11.275579 25046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance:
uuid: "3faa090f008a40789e985cc6f61fb53d"
format_stamp: "Formatted at 2026-08-11 17:20:11 on dist-test-slave-mjjr"
server_key: "5c71bcf76e69a3bd8369476bc54cc684"
server_key_iv: "2fb6813fa432aa8eddea12e88e4d4a94"
server_key_version: "encryptionkey@0"
I20260811 17:20:11.279981 25046 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:20:11.282851 25062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:11.284222 25046 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:20:11.284389 25046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "3faa090f008a40789e985cc6f61fb53d"
format_stamp: "Formatted at 2026-08-11 17:20:11 on dist-test-slave-mjjr"
server_key: "5c71bcf76e69a3bd8369476bc54cc684"
server_key_iv: "2fb6813fa432aa8eddea12e88e4d4a94"
server_key_version: "encryptionkey@0"
I20260811 17:20:11.284525 25046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:20:11.323381 25046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:20:11.324216 25046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:20:11.324448 25046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:20:11.325107 25046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:20:11.326143 25046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:20:11.326218 25046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:11.326294 25046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:20:11.326340 25046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:11.335999 25046 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:43023
I20260811 17:20:11.336024 25221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:43023 every 8 connection(s)
I20260811 17:20:11.337049 25046 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:20:11.341995 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 25046
I20260811 17:20:11.342130 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance
I20260811 17:20:11.342531 9846 external_mini_cluster.cc:1428] Setting key 765b96dd44438997a9436d41ef66ecae
I20260811 17:20:11.344837 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:0
--local_ip_for_outbound_sockets=127.9.157.131
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:40799
--builtin_ntp_servers=127.9.157.148:35917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 17:20:11.347734 24831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:11.339074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:48325 (local address 127.9.157.190:40799)
0811 17:20:11.339230 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:11.339234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:11.340522 (+ 1288us) server_negotiation.cc:407] Connection header received
0811 17:20:11.342232 (+ 1710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:11.342236 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:11.342301 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:11.342415 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:11.343598 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:11.344285 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:11.345593 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:11.345820 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:11.345933 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:11.347165 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:20:11.347169 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:20:11.347174 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:20:11.347193 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:20:11.347213 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:20:11.347221 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:20:11.347304 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 17:20:11.347485 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 17:20:11.347555 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 17:20:11.348759 25224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:11.339450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:40799 (local address 127.9.157.130:48325)
0811 17:20:11.340275 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:11.340392 (+ 117us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:11.341924 (+ 1532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:11.342656 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:11.342667 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:11.342780 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:11.343438 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:11.343459 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:11.344431 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:11.344439 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:11.345456 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:11.345467 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:11.346001 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:11.346008 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:11.346180 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:11.346863 (+ 683us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:20:11.346864 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:11.346911 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:20:11.346989 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:20:11.347312 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:20:11.347324 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:20:11.347332 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 17:20:11.347480 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 17:20:11.347860 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":217,"threads_started":1}
I20260811 17:20:11.350132 25222 heartbeater.cc:347] Connected to a master server at 127.9.157.190:40799
I20260811 17:20:11.350455 25222 heartbeater.cc:464] Registering TS with master...
I20260811 17:20:11.351280 25222 heartbeater.cc:511] Master 127.9.157.190:40799 requested a full tablet report, sending...
I20260811 17:20:11.352758 24775 ts_manager.cc:194] Registered new tserver with Master: 3faa090f008a40789e985cc6f61fb53d (127.9.157.130:43023)
I20260811 17:20:11.353699 24775 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.157.130:48325
W20260811 17:20:11.468858 25226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:20:11.469165 25226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:20:11.469204 25226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:20:11.473325 25226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:20:11.473397 25226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:20:11.473453 25226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:20:11.478389 25226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:35917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:40799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:20:11.479566 25226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:20:11.481033 25226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:20:11.491631 25232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:20:11.491619 25233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:20:11.491719 25226 server_base.cc:1034] running on GCE node
W20260811 17:20:11.491622 25236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:20:11.492380 25226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:20:11.493093 25226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:20:11.496387 25226 hybrid_clock.cc:648] HybridClock initialized: now 1786468811496344 us; error 50 us; skew 500 ppm
I20260811 17:20:11.498440 25226 webserver.cc:460] Webserver started at http://127.9.157.131:36787/ using document root <none> and password file <none>
I20260811 17:20:11.499102 25226 fs_manager.cc:359] Metadata directory not provided
I20260811 17:20:11.499174 25226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:20:11.499457 25226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:20:11.501737 25226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/data/instance:
uuid: "e1d205a3eca44e8f92eb6f25a95ed175"
format_stamp: "Formatted at 2026-08-11 17:20:11 on dist-test-slave-mjjr"
server_key: "a01fa153c4fde7de686e0e2c6a05472e"
server_key_iv: "71713d69fc6be179b18661f85a92139d"
server_key_version: "encryptionkey@0"
I20260811 17:20:11.502321 25226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance:
uuid: "e1d205a3eca44e8f92eb6f25a95ed175"
format_stamp: "Formatted at 2026-08-11 17:20:11 on dist-test-slave-mjjr"
server_key: "a01fa153c4fde7de686e0e2c6a05472e"
server_key_iv: "71713d69fc6be179b18661f85a92139d"
server_key_version: "encryptionkey@0"
I20260811 17:20:11.506354 25226 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 17:20:11.509212 25246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:11.510576 25226 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 17:20:11.510722 25226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "e1d205a3eca44e8f92eb6f25a95ed175"
format_stamp: "Formatted at 2026-08-11 17:20:11 on dist-test-slave-mjjr"
server_key: "a01fa153c4fde7de686e0e2c6a05472e"
server_key_iv: "71713d69fc6be179b18661f85a92139d"
server_key_version: "encryptionkey@0"
I20260811 17:20:11.510850 25226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:20:11.534315 25226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:20:11.535132 25226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:20:11.535359 25226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:20:11.536309 25226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:20:11.537410 25226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:20:11.537483 25226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:11.537551 25226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:20:11.537582 25226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:11.547008 25226 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:45691
I20260811 17:20:11.547026 25431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:45691 every 8 connection(s)
I20260811 17:20:11.548132 25226 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:20:11.553702 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 25226
I20260811 17:20:11.553829 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance
I20260811 17:20:11.554128 9846 external_mini_cluster.cc:1428] Setting key 8a358b79eed7cdf442442406402f6d04
I20260811 17:20:11.558521 24831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:11.550254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:34349 (local address 127.9.157.190:40799)
0811 17:20:11.550407 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:11.550411 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:11.551592 (+ 1181us) server_negotiation.cc:407] Connection header received
0811 17:20:11.553110 (+ 1518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:11.553113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:11.553175 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:11.553288 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:11.554535 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:11.555188 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:11.556686 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:11.556916 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:11.557005 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:11.558085 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:20:11.558088 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:20:11.558090 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:20:11.558103 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:20:11.558124 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:20:11.558132 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:20:11.558192 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 17:20:11.558336 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 17:20:11.558398 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 17:20:11.559430 25437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:11.550682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:40799 (local address 127.9.157.131:34349)
0811 17:20:11.551426 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:11.551469 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:11.552886 (+ 1417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:11.553492 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:11.553503 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:11.553617 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:11.554336 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:11.554370 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:11.555329 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:11.555336 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:11.556564 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:11.556574 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:11.557058 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:11.557060 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:11.557222 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:11.557820 (+ 598us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:20:11.557822 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:11.557866 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:20:11.557939 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:20:11.558207 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:20:11.558214 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:20:11.558219 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 17:20:11.558319 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 17:20:11.558565 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":126,"threads_started":1}
I20260811 17:20:11.560705 25433 heartbeater.cc:347] Connected to a master server at 127.9.157.190:40799
I20260811 17:20:11.560988 25433 heartbeater.cc:464] Registering TS with master...
I20260811 17:20:11.561580 25433 heartbeater.cc:511] Master 127.9.157.190:40799 requested a full tablet report, sending...
I20260811 17:20:11.562709 24775 ts_manager.cc:194] Registered new tserver with Master: e1d205a3eca44e8f92eb6f25a95ed175 (127.9.157.131:45691)
I20260811 17:20:11.563349 24775 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.157.131:34349
I20260811 17:20:11.568426 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 17:20:11.571550 24831 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:11.570544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39432 (local address 127.9.157.190:40799)
0811 17:20:11.570699 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:11.570704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:11.570891 (+ 187us) server_negotiation.cc:407] Connection header received
0811 17:20:11.571003 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:11.571005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:11.571049 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:11.571117 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:11.571413 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39432: BlockingRecv error: recv got EOF from 127.0.0.1:39432 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 17:20:11.571846 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 24860
I20260811 17:20:11.591377 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 25046
I20260811 17:20:11.605242 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 25226
I20260811 17:20:11.628326 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 24722
2026-08-11T17:20:11Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (1017 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T17:20:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:20:11Z Disabled control of system clock
I20260811 17:20:11.668784 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:40765
--webserver_interface=127.9.157.190
--webserver_port=0
--builtin_ntp_servers=127.9.157.148:43907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:40765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 17:20:11.787194 25450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:20:11.787499 25450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:20:11.787539 25450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:20:11.791661 25450 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:20:11.791740 25450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:20:11.791755 25450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:20:11.791774 25450 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:20:11.791788 25450 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:20:11.796794 25450 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:43907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:40765
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:40765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:20:11.798017 25450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:20:11.799484 25450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:20:11.808506 25458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:20:11.808521 25459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:20:11.808857 25450 server_base.cc:1034] running on GCE node
W20260811 17:20:11.808499 25461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:20:11.809566 25450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:20:11.810825 25450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:20:11.812031 25450 hybrid_clock.cc:648] HybridClock initialized: now 1786468811812008 us; error 39 us; skew 500 ppm
I20260811 17:20:11.814373 25450 webserver.cc:460] Webserver started at http://127.9.157.190:42207/ using document root <none> and password file <none>
I20260811 17:20:11.815066 25450 fs_manager.cc:359] Metadata directory not provided
I20260811 17:20:11.815173 25450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:20:11.815441 25450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:20:11.817935 25450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "d3700236d5d544759cc23cbbd1a3f35e"
format_stamp: "Formatted at 2026-08-11 17:20:11 on dist-test-slave-mjjr"
server_key: "ebd980628836700a51445dd48a1b1fae"
server_key_iv: "cdab12f5ee3d111ee22bb66037a56c43"
server_key_version: "encryptionkey@0"
I20260811 17:20:11.818666 25450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "d3700236d5d544759cc23cbbd1a3f35e"
format_stamp: "Formatted at 2026-08-11 17:20:11 on dist-test-slave-mjjr"
server_key: "ebd980628836700a51445dd48a1b1fae"
server_key_iv: "cdab12f5ee3d111ee22bb66037a56c43"
server_key_version: "encryptionkey@0"
I20260811 17:20:11.823124 25450 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:20:11.826081 25470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:11.827440 25450 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:20:11.827610 25450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "d3700236d5d544759cc23cbbd1a3f35e"
format_stamp: "Formatted at 2026-08-11 17:20:11 on dist-test-slave-mjjr"
server_key: "ebd980628836700a51445dd48a1b1fae"
server_key_iv: "cdab12f5ee3d111ee22bb66037a56c43"
server_key_version: "encryptionkey@0"
I20260811 17:20:11.827809 25450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:20:11.835711 25450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:20:11.836491 25450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:20:11.836704 25450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:20:11.844182 25450 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:40765
I20260811 17:20:11.844203 25553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:40765 every 8 connection(s)
I20260811 17:20:11.845265 25450 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:20:11.845687 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 25450
I20260811 17:20:11.845830 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:20:11.846186 9846 external_mini_cluster.cc:1428] Setting key c1f3aa48a21c5a207b6e77fea0313584
I20260811 17:20:11.849099 25554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:20:11.852910 25557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:11.847795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41968 (local address 127.9.157.190:40765)
0811 17:20:11.849126 (+ 1331us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:11.849144 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:11.849245 (+ 101us) server_negotiation.cc:407] Connection header received
0811 17:20:11.849951 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:11.849969 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:11.850352 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:11.850731 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:11.851544 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:20:11.851549 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:20:11.851562 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:20:11.851571 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:20:11.851622 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:20:11.851695 (+ 73us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:20:11.851878 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 17:20:11.851948 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 17:20:11.852197 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":944,"thread_start_us":736,"threads_started":1}
I20260811 17:20:11.855401 25554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3700236d5d544759cc23cbbd1a3f35e: Bootstrap starting.
I20260811 17:20:11.857895 25554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3700236d5d544759cc23cbbd1a3f35e: Neither blocks nor log segments found. Creating new log.
I20260811 17:20:11.858693 25554 log.cc:824] T 00000000000000000000000000000000 P d3700236d5d544759cc23cbbd1a3f35e: Log is configured to *not* fsync() on all Append() calls
I20260811 17:20:11.860846 25554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3700236d5d544759cc23cbbd1a3f35e: No bootstrap required, opened a new log
I20260811 17:20:11.865115 25554 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3700236d5d544759cc23cbbd1a3f35e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3700236d5d544759cc23cbbd1a3f35e" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 40765 } }
I20260811 17:20:11.865461 25554 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3700236d5d544759cc23cbbd1a3f35e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:20:11.865532 25554 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3700236d5d544759cc23cbbd1a3f35e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3700236d5d544759cc23cbbd1a3f35e, State: Initialized, Role: FOLLOWER
I20260811 17:20:11.866103 25554 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3700236d5d544759cc23cbbd1a3f35e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3700236d5d544759cc23cbbd1a3f35e" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 40765 } }
I20260811 17:20:11.866297 25554 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3700236d5d544759cc23cbbd1a3f35e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:20:11.866375 25554 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3700236d5d544759cc23cbbd1a3f35e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:20:11.866492 25554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3700236d5d544759cc23cbbd1a3f35e [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:20:11.868016 25554 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3700236d5d544759cc23cbbd1a3f35e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3700236d5d544759cc23cbbd1a3f35e" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 40765 } }
I20260811 17:20:11.868451 25554 leader_election.cc:302] T 00000000000000000000000000000000 P d3700236d5d544759cc23cbbd1a3f35e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3700236d5d544759cc23cbbd1a3f35e; no voters:
I20260811 17:20:11.869489 25554 leader_election.cc:288] T 00000000000000000000000000000000 P d3700236d5d544759cc23cbbd1a3f35e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:20:11.869645 25561 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3700236d5d544759cc23cbbd1a3f35e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:20:11.869918 25561 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3700236d5d544759cc23cbbd1a3f35e [term 1 LEADER]: Becoming Leader. State: Replica: d3700236d5d544759cc23cbbd1a3f35e, State: Running, Role: LEADER
I20260811 17:20:11.870277 25561 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3700236d5d544759cc23cbbd1a3f35e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3700236d5d544759cc23cbbd1a3f35e" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 40765 } }
I20260811 17:20:11.870985 25554 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3700236d5d544759cc23cbbd1a3f35e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:20:11.873018 25567 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3700236d5d544759cc23cbbd1a3f35e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3700236d5d544759cc23cbbd1a3f35e. Latest consensus state: current_term: 1 leader_uuid: "d3700236d5d544759cc23cbbd1a3f35e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3700236d5d544759cc23cbbd1a3f35e" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 40765 } } }
I20260811 17:20:11.873167 25567 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3700236d5d544759cc23cbbd1a3f35e [sys.catalog]: This master's current role is: LEADER
I20260811 17:20:11.874394 25565 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3700236d5d544759cc23cbbd1a3f35e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3700236d5d544759cc23cbbd1a3f35e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3700236d5d544759cc23cbbd1a3f35e" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 40765 } } }
I20260811 17:20:11.874754 25565 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3700236d5d544759cc23cbbd1a3f35e [sys.catalog]: This master's current role is: LEADER
W20260811 17:20:11.877173 25588 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d3700236d5d544759cc23cbbd1a3f35e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:20:11.877244 25588 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:20:11.877326 25589 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:20:11.878054 25589 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:20:11.883632 25589 catalog_manager.cc:1347] Generated new cluster ID: f4ae5262b1ea4d45b32b88e091b968f2
I20260811 17:20:11.883746 25589 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:20:11.911690 25589 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:20:11.912098 25589 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:20:11.927495 25589 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3700236d5d544759cc23cbbd1a3f35e: Generated new TSK 0
I20260811 17:20:11.928423 25589 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:20:11.942232 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:0
--local_ip_for_outbound_sockets=127.9.157.129
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:40765
--builtin_ntp_servers=127.9.157.148:43907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 17:20:12.067081 25596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:20:12.067415 25596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:20:12.067461 25596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:20:12.071878 25596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:20:12.071985 25596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:20:12.072074 25596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:20:12.077306 25596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:43907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:40765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:20:12.078585 25596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:20:12.080183 25596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:20:12.090996 25609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:20:12.090996 25602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:20:12.090981 25603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:20:12.091365 25596 server_base.cc:1034] running on GCE node
I20260811 17:20:12.092026 25596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:20:12.092758 25596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:20:12.093961 25596 hybrid_clock.cc:648] HybridClock initialized: now 1786468812093922 us; error 54 us; skew 500 ppm
I20260811 17:20:12.096452 25596 webserver.cc:460] Webserver started at http://127.9.157.129:46305/ using document root <none> and password file <none>
I20260811 17:20:12.097223 25596 fs_manager.cc:359] Metadata directory not provided
I20260811 17:20:12.097327 25596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:20:12.097564 25596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:20:12.099988 25596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/data/instance:
uuid: "bd1d479f66d345cda8ce91880e1754ba"
format_stamp: "Formatted at 2026-08-11 17:20:12 on dist-test-slave-mjjr"
server_key: "ee3943856b3a174baa62bb44e4a67054"
server_key_iv: "3a0e061cc18a6c4c3baa757062dd43bd"
server_key_version: "encryptionkey@0"
I20260811 17:20:12.100713 25596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance:
uuid: "bd1d479f66d345cda8ce91880e1754ba"
format_stamp: "Formatted at 2026-08-11 17:20:12 on dist-test-slave-mjjr"
server_key: "ee3943856b3a174baa62bb44e4a67054"
server_key_iv: "3a0e061cc18a6c4c3baa757062dd43bd"
server_key_version: "encryptionkey@0"
I20260811 17:20:12.105317 25596 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:20:12.108446 25616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:12.109942 25596 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 17:20:12.110122 25596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "bd1d479f66d345cda8ce91880e1754ba"
format_stamp: "Formatted at 2026-08-11 17:20:12 on dist-test-slave-mjjr"
server_key: "ee3943856b3a174baa62bb44e4a67054"
server_key_iv: "3a0e061cc18a6c4c3baa757062dd43bd"
server_key_version: "encryptionkey@0"
I20260811 17:20:12.110267 25596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:20:12.115476 25596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:20:12.116263 25596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:20:12.116477 25596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:20:12.117112 25596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:20:12.118171 25596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:20:12.118249 25596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:12.118323 25596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:20:12.118371 25596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:12.128103 25596 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:42089
I20260811 17:20:12.128120 25787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:42089 every 8 connection(s)
I20260811 17:20:12.129160 25596 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:20:12.130749 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 25596
I20260811 17:20:12.130905 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance
I20260811 17:20:12.131266 9846 external_mini_cluster.cc:1428] Setting key c41369af41103d618048916ece8c5a7e
I20260811 17:20:12.133663 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:0
--local_ip_for_outbound_sockets=127.9.157.130
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:40765
--builtin_ntp_servers=127.9.157.148:43907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 17:20:12.136444 25557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:12.131224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:51607 (local address 127.9.157.190:40765)
0811 17:20:12.131374 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:12.131380 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:12.133015 (+ 1635us) server_negotiation.cc:407] Connection header received
0811 17:20:12.134458 (+ 1443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:12.134464 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:12.134517 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:12.134614 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:12.135970 (+ 1356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:20:12.135974 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:20:12.135976 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:20:12.135988 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:20:12.136010 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:20:12.136018 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:20:12.136096 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 17:20:12.136273 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 17:20:12.136315 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 17:20:12.137574 25791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:12.131884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:40765 (local address 127.9.157.129:51607)
0811 17:20:12.132815 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:12.132884 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:12.134237 (+ 1353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:12.134780 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:12.134787 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:12.134897 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:12.135564 (+ 667us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:20:12.135568 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:12.135663 (+ 95us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:20:12.135773 (+ 110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:20:12.136108 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:20:12.136120 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:20:12.136128 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 17:20:12.136325 (+ 197us) client_negotiation.cc:240] Negotiation successful
0811 17:20:12.136605 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":190,"threads_started":1}
I20260811 17:20:12.138952 25788 heartbeater.cc:347] Connected to a master server at 127.9.157.190:40765
I20260811 17:20:12.139223 25788 heartbeater.cc:464] Registering TS with master...
I20260811 17:20:12.139837 25788 heartbeater.cc:511] Master 127.9.157.190:40765 requested a full tablet report, sending...
I20260811 17:20:12.141904 25487 ts_manager.cc:194] Registered new tserver with Master: bd1d479f66d345cda8ce91880e1754ba (127.9.157.129:42089)
W20260811 17:20:12.254998 25792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:20:12.255285 25792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:20:12.255328 25792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:20:12.259423 25792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:20:12.259496 25792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:20:12.259555 25792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:20:12.264400 25792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:43907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:40765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:20:12.265544 25792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:20:12.267004 25792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:20:12.277966 25799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:20:12.277966 25802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:20:12.278059 25800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:20:12.278326 25792 server_base.cc:1034] running on GCE node
I20260811 17:20:12.278828 25792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:20:12.279528 25792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:20:12.280737 25792 hybrid_clock.cc:648] HybridClock initialized: now 1786468812280723 us; error 30 us; skew 500 ppm
I20260811 17:20:12.287592 25792 webserver.cc:460] Webserver started at http://127.9.157.130:39423/ using document root <none> and password file <none>
I20260811 17:20:12.288305 25792 fs_manager.cc:359] Metadata directory not provided
I20260811 17:20:12.288421 25792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:20:12.288683 25792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:20:12.290969 25792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/data/instance:
uuid: "6babf9c73011492b8a02508e68cc7756"
format_stamp: "Formatted at 2026-08-11 17:20:12 on dist-test-slave-mjjr"
server_key: "129d1e5023e99178cd60c59c23532896"
server_key_iv: "5f97d902ce00e90771c9e4e9a8849ca7"
server_key_version: "encryptionkey@0"
I20260811 17:20:12.291586 25792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance:
uuid: "6babf9c73011492b8a02508e68cc7756"
format_stamp: "Formatted at 2026-08-11 17:20:12 on dist-test-slave-mjjr"
server_key: "129d1e5023e99178cd60c59c23532896"
server_key_iv: "5f97d902ce00e90771c9e4e9a8849ca7"
server_key_version: "encryptionkey@0"
I20260811 17:20:12.295694 25792 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260811 17:20:12.298502 25811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:12.299907 25792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:20:12.300047 25792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "6babf9c73011492b8a02508e68cc7756"
format_stamp: "Formatted at 2026-08-11 17:20:12 on dist-test-slave-mjjr"
server_key: "129d1e5023e99178cd60c59c23532896"
server_key_iv: "5f97d902ce00e90771c9e4e9a8849ca7"
server_key_version: "encryptionkey@0"
I20260811 17:20:12.300174 25792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:20:12.305202 25792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:20:12.305927 25792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:20:12.306135 25792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:20:12.306770 25792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:20:12.307883 25792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:20:12.307955 25792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:12.308032 25792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:20:12.308068 25792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:12.318257 25792 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:37723
I20260811 17:20:12.318281 25969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:37723 every 8 connection(s)
I20260811 17:20:12.319257 25792 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:20:12.323760 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 25792
I20260811 17:20:12.323910 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance
I20260811 17:20:12.324275 9846 external_mini_cluster.cc:1428] Setting key 38b7347a09c3bb52e74aefb6097902bc
I20260811 17:20:12.326333 25557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:12.321262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:38343 (local address 127.9.157.190:40765)
0811 17:20:12.321449 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:12.321454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:12.322604 (+ 1150us) server_negotiation.cc:407] Connection header received
0811 17:20:12.324072 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:12.324076 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:12.324137 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:12.324244 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:12.325708 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:20:12.325714 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:20:12.325716 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:20:12.325761 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:20:12.325793 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:20:12.325808 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:20:12.325912 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 17:20:12.326121 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 17:20:12.326174 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 17:20:12.326848 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:0
--local_ip_for_outbound_sockets=127.9.157.131
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:40765
--builtin_ntp_servers=127.9.157.148:43907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 17:20:12.327328 25972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:12.321735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:40765 (local address 127.9.157.130:38343)
0811 17:20:12.322448 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:12.322491 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:12.323793 (+ 1302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:12.324460 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:12.324468 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:12.324563 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:12.325340 (+ 777us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:20:12.325342 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:12.325424 (+ 82us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:20:12.325520 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:20:12.325922 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:20:12.325936 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:20:12.325945 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 17:20:12.326149 (+ 204us) client_negotiation.cc:240] Negotiation successful
0811 17:20:12.326374 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":135,"threads_started":1}
I20260811 17:20:12.328693 25970 heartbeater.cc:347] Connected to a master server at 127.9.157.190:40765
I20260811 17:20:12.328970 25970 heartbeater.cc:464] Registering TS with master...
I20260811 17:20:12.329463 25970 heartbeater.cc:511] Master 127.9.157.190:40765 requested a full tablet report, sending...
I20260811 17:20:12.330626 25487 ts_manager.cc:194] Registered new tserver with Master: 6babf9c73011492b8a02508e68cc7756 (127.9.157.130:37723)
W20260811 17:20:12.451236 25973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:20:12.451551 25973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:20:12.451593 25973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:20:12.455912 25973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:20:12.455992 25973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:20:12.456084 25973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:20:12.461306 25973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:43907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:40765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:20:12.462559 25973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:20:12.464069 25973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:20:12.474151 25981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:20:12.474186 25980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:20:12.474326 25983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:20:12.474962 25973 server_base.cc:1034] running on GCE node
I20260811 17:20:12.475414 25973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:20:12.476194 25973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:20:12.477393 25973 hybrid_clock.cc:648] HybridClock initialized: now 1786468812477381 us; error 42 us; skew 500 ppm
I20260811 17:20:12.479307 25973 webserver.cc:460] Webserver started at http://127.9.157.131:33445/ using document root <none> and password file <none>
I20260811 17:20:12.480021 25973 fs_manager.cc:359] Metadata directory not provided
I20260811 17:20:12.480118 25973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:20:12.480368 25973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:20:12.482540 25973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/data/instance:
uuid: "cc4233a6befb41e1b9356cce1c6596ec"
format_stamp: "Formatted at 2026-08-11 17:20:12 on dist-test-slave-mjjr"
server_key: "496bf5b96c6dded453848699b2c5a497"
server_key_iv: "6ad2cffd8e5ce412e1be809a0685d928"
server_key_version: "encryptionkey@0"
I20260811 17:20:12.483152 25973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance:
uuid: "cc4233a6befb41e1b9356cce1c6596ec"
format_stamp: "Formatted at 2026-08-11 17:20:12 on dist-test-slave-mjjr"
server_key: "496bf5b96c6dded453848699b2c5a497"
server_key_iv: "6ad2cffd8e5ce412e1be809a0685d928"
server_key_version: "encryptionkey@0"
I20260811 17:20:12.487125 25973 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260811 17:20:12.489813 25992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:12.491210 25973 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 17:20:12.491382 25973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "cc4233a6befb41e1b9356cce1c6596ec"
format_stamp: "Formatted at 2026-08-11 17:20:12 on dist-test-slave-mjjr"
server_key: "496bf5b96c6dded453848699b2c5a497"
server_key_iv: "6ad2cffd8e5ce412e1be809a0685d928"
server_key_version: "encryptionkey@0"
I20260811 17:20:12.491511 25973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:20:12.496968 25973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:20:12.497661 25973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:20:12.497869 25973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:20:12.498471 25973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:20:12.499441 25973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:20:12.499511 25973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:12.499577 25973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:20:12.499624 25973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:12.508841 25973 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:40037
I20260811 17:20:12.508913 26144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:40037 every 8 connection(s)
I20260811 17:20:12.509864 25973 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:20:12.516069 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 25973
I20260811 17:20:12.516188 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance
I20260811 17:20:12.516522 9846 external_mini_cluster.cc:1428] Setting key 6341df934647f4fe79aeacb398ef8ebd
I20260811 17:20:12.516903 25557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:12.511935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:36619 (local address 127.9.157.190:40765)
0811 17:20:12.512091 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:12.512095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:12.513400 (+ 1305us) server_negotiation.cc:407] Connection header received
0811 17:20:12.514878 (+ 1478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:12.514881 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:12.514929 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:12.515040 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:12.516352 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:20:12.516356 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:20:12.516358 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:20:12.516371 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:20:12.516392 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:20:12.516401 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:20:12.516490 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 17:20:12.516713 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 17:20:12.516759 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 17:20:12.518079 26150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:12.512419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:40765 (local address 127.9.157.131:36619)
0811 17:20:12.513228 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:12.513270 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:12.514650 (+ 1380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:12.515217 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:12.515224 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:12.515313 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:12.516009 (+ 696us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:20:12.516012 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:12.516071 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:20:12.516166 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:20:12.516497 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:20:12.516507 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:20:12.516514 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 17:20:12.516756 (+ 242us) client_negotiation.cc:240] Negotiation successful
0811 17:20:12.517048 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":164,"threads_started":1}
I20260811 17:20:12.519253 26145 heartbeater.cc:347] Connected to a master server at 127.9.157.190:40765
I20260811 17:20:12.519502 26145 heartbeater.cc:464] Registering TS with master...
I20260811 17:20:12.519987 26145 heartbeater.cc:511] Master 127.9.157.190:40765 requested a full tablet report, sending...
I20260811 17:20:12.521044 25487 ts_manager.cc:194] Registered new tserver with Master: cc4233a6befb41e1b9356cce1c6596ec (127.9.157.131:40037)
I20260811 17:20:12.530721 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 17:20:12.534015 25557 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:20:12.533064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41978 (local address 127.9.157.190:40765)
0811 17:20:12.533193 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:12.533197 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:12.533386 (+ 189us) server_negotiation.cc:407] Connection header received
0811 17:20:12.533491 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:12.533493 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:12.533527 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:12.533595 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:12.533871 (+ 276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41978: BlockingRecv error: recv got EOF from 127.0.0.1:41978 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 17:20:12.534289 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 25596
I20260811 17:20:12.560351 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 25792
I20260811 17:20:12.586148 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 25973
I20260811 17:20:12.612142 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 25450
2026-08-11T17:20:12Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (983 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:20:12 dist-test-slave-mjjr krb5kdc[26163](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:20:12 dist-test-slave-mjjr krb5kdc[26163](info): set up 2 sockets
Aug 11 17:20:12 dist-test-slave-mjjr krb5kdc[26163](info): commencing operation
krb5kdc: starting...
W20260811 17:20:13.896934 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:20:13 dist-test-slave-mjjr krb5kdc[26163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:20:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:20:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:20:14.034978 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:37835
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:37835
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:20:14.157473 26179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:20:14.157904 26179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:20:14.157994 26179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:20:14.165903 26179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:20:14.165977 26179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:20:14.165999 26179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:20:14.166018 26179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:20:14.166033 26179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:20:14.171958 26179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:37835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:37835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:20:14.173122 26179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:20:14.174255 26179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:20:14.183446 26179 server_base.cc:1034] running on GCE node
W20260811 17:20:14.183677 26187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:20:14.183715 26191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:20:14.183729 26185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:20:14.184469 26179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:20:14.185676 26179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:20:14.186872 26179 hybrid_clock.cc:648] HybridClock initialized: now 1786468814186854 us; error 35 us; skew 500 ppm
Aug 11 17:20:14 dist-test-slave-mjjr krb5kdc[26163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468814, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:20:14.189783 26179 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:20:14.190780 26179 webserver.cc:460] Webserver started at http://127.9.157.190:35013/ using document root <none> and password file <none>
I20260811 17:20:14.191380 26179 fs_manager.cc:359] Metadata directory not provided
I20260811 17:20:14.191475 26179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:20:14.191787 26179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:20:14.194020 26179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "82003876dc9046f49f786a1a961d9e81"
format_stamp: "Formatted at 2026-08-11 17:20:14 on dist-test-slave-mjjr"
server_key: "9412b52bfaf2e4012b6c5b55d27b1d6b"
server_key_iv: "f635a009ef00f4f6136d098114ff39c9"
server_key_version: "encryptionkey@0"
I20260811 17:20:14.194633 26179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "82003876dc9046f49f786a1a961d9e81"
format_stamp: "Formatted at 2026-08-11 17:20:14 on dist-test-slave-mjjr"
server_key: "9412b52bfaf2e4012b6c5b55d27b1d6b"
server_key_iv: "f635a009ef00f4f6136d098114ff39c9"
server_key_version: "encryptionkey@0"
I20260811 17:20:14.198611 26179 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 17:20:14.201267 26199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:14.202502 26179 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 17:20:14.202656 26179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "82003876dc9046f49f786a1a961d9e81"
format_stamp: "Formatted at 2026-08-11 17:20:14 on dist-test-slave-mjjr"
server_key: "9412b52bfaf2e4012b6c5b55d27b1d6b"
server_key_iv: "f635a009ef00f4f6136d098114ff39c9"
server_key_version: "encryptionkey@0"
I20260811 17:20:14.202776 26179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:20:14.221642 26179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:20:14.224957 26179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:20:14.225184 26179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:20:14.233070 26179 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:37835
I20260811 17:20:14.233385 26272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:37835 every 8 connection(s)
I20260811 17:20:14.234160 26179 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:20:14.234766 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 26179
I20260811 17:20:14.234891 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:20:14.235188 9846 external_mini_cluster.cc:1428] Setting key be389f01d0d8ce2b0146717ff8513741
I20260811 17:20:14.237663 26273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 17:20:14 dist-test-slave-mjjr krb5kdc[26163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:20:14.244957 26273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82003876dc9046f49f786a1a961d9e81: Bootstrap starting.
I20260811 17:20:14.248080 26273 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82003876dc9046f49f786a1a961d9e81: Neither blocks nor log segments found. Creating new log.
I20260811 17:20:14.249096 26273 log.cc:824] T 00000000000000000000000000000000 P 82003876dc9046f49f786a1a961d9e81: Log is configured to *not* fsync() on all Append() calls
I20260811 17:20:14.250993 26276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:14.236748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57566 (local address 127.9.157.190:37835)
0811 17:20:14.237196 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:14.237216 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:14.237270 (+ 54us) server_negotiation.cc:407] Connection header received
0811 17:20:14.238153 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:14.238178 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:14.238613 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:14.238984 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:14.239742 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:14.241288 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:14.242150 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:14.242841 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:14.243026 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:14.245612 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:20:14.245619 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:20:14.245635 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:20:14.245668 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:20:14.247686 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:20:14.248131 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:20:14.248139 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:20:14.248147 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:20:14.248295 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:20:14.248574 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:20:14.248577 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:20:14.248579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:20:14.249118 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:20:14.249317 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 17:20:14.249608 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 17:20:14.249961 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":115,"threads_started":1}
I20260811 17:20:14.251560 26273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82003876dc9046f49f786a1a961d9e81: No bootstrap required, opened a new log
I20260811 17:20:14.255414 26273 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82003876dc9046f49f786a1a961d9e81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82003876dc9046f49f786a1a961d9e81" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37835 } }
I20260811 17:20:14.255710 26273 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82003876dc9046f49f786a1a961d9e81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:20:14.255801 26273 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82003876dc9046f49f786a1a961d9e81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82003876dc9046f49f786a1a961d9e81, State: Initialized, Role: FOLLOWER
I20260811 17:20:14.256258 26273 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82003876dc9046f49f786a1a961d9e81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82003876dc9046f49f786a1a961d9e81" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37835 } }
I20260811 17:20:14.256408 26273 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82003876dc9046f49f786a1a961d9e81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:20:14.256475 26273 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82003876dc9046f49f786a1a961d9e81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:20:14.256577 26273 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82003876dc9046f49f786a1a961d9e81 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:20:14.257656 26273 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82003876dc9046f49f786a1a961d9e81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82003876dc9046f49f786a1a961d9e81" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37835 } }
I20260811 17:20:14.258035 26273 leader_election.cc:302] T 00000000000000000000000000000000 P 82003876dc9046f49f786a1a961d9e81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82003876dc9046f49f786a1a961d9e81; no voters:
I20260811 17:20:14.258409 26273 leader_election.cc:288] T 00000000000000000000000000000000 P 82003876dc9046f49f786a1a961d9e81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:20:14.258579 26278 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82003876dc9046f49f786a1a961d9e81 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:20:14.258852 26278 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82003876dc9046f49f786a1a961d9e81 [term 1 LEADER]: Becoming Leader. State: Replica: 82003876dc9046f49f786a1a961d9e81, State: Running, Role: LEADER
I20260811 17:20:14.259177 26278 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82003876dc9046f49f786a1a961d9e81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82003876dc9046f49f786a1a961d9e81" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37835 } }
I20260811 17:20:14.259918 26273 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82003876dc9046f49f786a1a961d9e81 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:20:14.261240 26279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82003876dc9046f49f786a1a961d9e81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82003876dc9046f49f786a1a961d9e81. Latest consensus state: current_term: 1 leader_uuid: "82003876dc9046f49f786a1a961d9e81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82003876dc9046f49f786a1a961d9e81" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37835 } } }
I20260811 17:20:14.261250 26282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82003876dc9046f49f786a1a961d9e81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82003876dc9046f49f786a1a961d9e81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82003876dc9046f49f786a1a961d9e81" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37835 } } }
I20260811 17:20:14.261507 26279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82003876dc9046f49f786a1a961d9e81 [sys.catalog]: This master's current role is: LEADER
I20260811 17:20:14.261543 26282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82003876dc9046f49f786a1a961d9e81 [sys.catalog]: This master's current role is: LEADER
I20260811 17:20:14.262072 26296 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:20:14.265617 26296 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:20:14.271544 26296 catalog_manager.cc:1347] Generated new cluster ID: ab4d86bda21944f18ff46d25cfc08db0
I20260811 17:20:14.271646 26296 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:20:14.286356 26296 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:20:14.287418 26296 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:20:14.296802 26296 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82003876dc9046f49f786a1a961d9e81: Generated new TSK 0
I20260811 17:20:14.297618 26296 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
I20260811 17:20:14.332985 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:0
--local_ip_for_outbound_sockets=127.9.157.129
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:37835
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:20:14.460259 26312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:20:14.460620 26312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:20:14.460668 26312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:20:14.464841 26312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:20:14.464918 26312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:20:14.464982 26312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:20:14.470258 26312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:37835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:20:14.471388 26312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:20:14.472417 26312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:20:14.482560 26319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:20:14.482574 26312 server_base.cc:1034] running on GCE node
W20260811 17:20:14.482573 26321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:20:14.482616 26323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:20:14.483253 26312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:20:14.483971 26312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:20:14.485142 26312 hybrid_clock.cc:648] HybridClock initialized: now 1786468814485123 us; error 35 us; skew 500 ppm
Aug 11 17:20:14 dist-test-slave-mjjr krb5kdc[26163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468814, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:20:14.488300 26312 init.cc:377] Logged in from keytab as kudu/127.9.157.129@KRBTEST.COM (short username kudu)
I20260811 17:20:14.489240 26312 webserver.cc:460] Webserver started at http://127.9.157.129:38619/ using document root <none> and password file <none>
I20260811 17:20:14.489863 26312 fs_manager.cc:359] Metadata directory not provided
I20260811 17:20:14.489971 26312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:20:14.490224 26312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:20:14.492519 26312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/data/instance:
uuid: "fed7ff6acbac4e5ca9e5da9ae0a8e173"
format_stamp: "Formatted at 2026-08-11 17:20:14 on dist-test-slave-mjjr"
server_key: "bb76cb764e019003db95c38e1d068a5c"
server_key_iv: "636f2670c4c0c3e55775fdbc6e4a4291"
server_key_version: "encryptionkey@0"
I20260811 17:20:14.493120 26312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance:
uuid: "fed7ff6acbac4e5ca9e5da9ae0a8e173"
format_stamp: "Formatted at 2026-08-11 17:20:14 on dist-test-slave-mjjr"
server_key: "bb76cb764e019003db95c38e1d068a5c"
server_key_iv: "636f2670c4c0c3e55775fdbc6e4a4291"
server_key_version: "encryptionkey@0"
I20260811 17:20:14.497169 26312 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:20:14.499903 26332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:14.501228 26312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:20:14.501389 26312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "fed7ff6acbac4e5ca9e5da9ae0a8e173"
format_stamp: "Formatted at 2026-08-11 17:20:14 on dist-test-slave-mjjr"
server_key: "bb76cb764e019003db95c38e1d068a5c"
server_key_iv: "636f2670c4c0c3e55775fdbc6e4a4291"
server_key_version: "encryptionkey@0"
I20260811 17:20:14.501518 26312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:20:14.532559 26312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:20:14.535915 26312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:20:14.536144 26312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:20:14.536798 26312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:20:14.537736 26312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:20:14.537813 26312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:14.537910 26312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:20:14.537962 26312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:14.547428 26312 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:42895
I20260811 17:20:14.547453 26508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:42895 every 8 connection(s)
I20260811 17:20:14.548442 26312 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:20:14.552322 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 26312
I20260811 17:20:14.552500 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance
I20260811 17:20:14.552850 9846 external_mini_cluster.cc:1428] Setting key 915ce15c642bba29f1bfe9a4372ca076
Aug 11 17:20:14 dist-test-slave-mjjr krb5kdc[26163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468814, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.157.130@KRBTEST.COM; defaulting to no policy
I20260811 17:20:14.564980 26276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:14.550571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:60043 (local address 127.9.157.190:37835)
0811 17:20:14.550728 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:14.550732 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:14.551629 (+ 897us) server_negotiation.cc:407] Connection header received
0811 17:20:14.553582 (+ 1953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:14.553586 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:14.553639 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:14.553788 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:14.555425 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:14.556079 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:14.557555 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:14.557792 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:14.557905 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:14.560835 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:20:14.560838 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:20:14.560841 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:20:14.560870 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:20:14.562525 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:20:14.563095 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:20:14.563098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:20:14.563100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:20:14.563188 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:20:14.563571 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:20:14.563576 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:20:14.563578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:20:14.563882 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:20:14.564009 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 17:20:14.564630 (+ 621us) server_negotiation.cc:299] Negotiation successful
0811 17:20:14.564781 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 17:20:14.565939 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:14.550916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:37835 (local address 127.9.157.129:60043)
0811 17:20:14.551452 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:14.551500 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:14.553313 (+ 1813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:14.553951 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:14.553958 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:14.554490 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:14.555235 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:14.555259 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:14.556226 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:14.556233 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:14.557404 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:14.557416 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:14.557980 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:14.557984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:14.558159 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:14.558807 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:14.558825 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:14.560638 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:20:14.562661 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:20:14.562668 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:20:14.562688 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:20:14.562974 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:20:14.563288 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:20:14.563291 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:20:14.563293 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:20:14.563465 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:20:14.564009 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:20:14.564018 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:20:14.564314 (+ 296us) client_negotiation.cc:769] Sending connection context
0811 17:20:14.564532 (+ 218us) client_negotiation.cc:240] Negotiation successful
0811 17:20:14.564914 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":134,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.130@KRBTEST.COM" created.
I20260811 17:20:14.567994 26509 heartbeater.cc:347] Connected to a master server at 127.9.157.190:37835
I20260811 17:20:14.568346 26509 heartbeater.cc:464] Registering TS with master...
I20260811 17:20:14.568959 26509 heartbeater.cc:511] Master 127.9.157.190:37835 requested a full tablet report, sending...
I20260811 17:20:14.571038 26225 ts_manager.cc:194] Registered new tserver with Master: fed7ff6acbac4e5ca9e5da9ae0a8e173 (127.9.157.129:42895)
I20260811 17:20:14.572887 26225 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.129@KRBTEST.COM'} at 127.9.157.129:60043
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
I20260811 17:20:14.587316 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:0
--local_ip_for_outbound_sockets=127.9.157.130
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:37835
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:20:14.711162 26518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:20:14.711519 26518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:20:14.711601 26518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:20:14.715771 26518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:20:14.715876 26518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:20:14.715999 26518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:20:14.721625 26518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:37835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:20:14.722776 26518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:20:14.723850 26518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:20:14.734282 26530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:20:14.734263 26528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:20:14.734483 26518 server_base.cc:1034] running on GCE node
W20260811 17:20:14.734257 26526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:20:14.735136 26518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:20:14.735929 26518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:20:14.737125 26518 hybrid_clock.cc:648] HybridClock initialized: now 1786468814737110 us; error 29 us; skew 500 ppm
Aug 11 17:20:14 dist-test-slave-mjjr krb5kdc[26163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468814, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:20:14.748220 26518 init.cc:377] Logged in from keytab as kudu/127.9.157.130@KRBTEST.COM (short username kudu)
I20260811 17:20:14.749325 26518 webserver.cc:460] Webserver started at http://127.9.157.130:38317/ using document root <none> and password file <none>
I20260811 17:20:14.749991 26518 fs_manager.cc:359] Metadata directory not provided
I20260811 17:20:14.750097 26518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:20:14.750332 26518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:20:14.752724 26518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/data/instance:
uuid: "718afe47844b4f6dacee280d5cfbd117"
format_stamp: "Formatted at 2026-08-11 17:20:14 on dist-test-slave-mjjr"
server_key: "47891b654cca171df3a64b37d65c2660"
server_key_iv: "bdf6e6278d1550327fef021981380977"
server_key_version: "encryptionkey@0"
I20260811 17:20:14.753355 26518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance:
uuid: "718afe47844b4f6dacee280d5cfbd117"
format_stamp: "Formatted at 2026-08-11 17:20:14 on dist-test-slave-mjjr"
server_key: "47891b654cca171df3a64b37d65c2660"
server_key_iv: "bdf6e6278d1550327fef021981380977"
server_key_version: "encryptionkey@0"
I20260811 17:20:14.757723 26518 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:20:14.760957 26543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:14.762486 26518 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 17:20:14.762598 26518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "718afe47844b4f6dacee280d5cfbd117"
format_stamp: "Formatted at 2026-08-11 17:20:14 on dist-test-slave-mjjr"
server_key: "47891b654cca171df3a64b37d65c2660"
server_key_iv: "bdf6e6278d1550327fef021981380977"
server_key_version: "encryptionkey@0"
I20260811 17:20:14.762703 26518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:20:14.782043 26518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:20:14.785216 26518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:20:14.785391 26518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:20:14.786051 26518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:20:14.786944 26518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:20:14.786996 26518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:14.787038 26518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:20:14.787055 26518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:14.796955 26518 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:35231
I20260811 17:20:14.797017 26710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:35231 every 8 connection(s)
I20260811 17:20:14.797971 26518 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:20:14.806571 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 26518
I20260811 17:20:14.806700 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance
I20260811 17:20:14.807102 9846 external_mini_cluster.cc:1428] Setting key 6da3314f66e03d37d98c611dfc760c4a
Aug 11 17:20:14 dist-test-slave-mjjr krb5kdc[26163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468814, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:20:14.815320 26276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:14.800079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:50889 (local address 127.9.157.190:37835)
0811 17:20:14.800241 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:14.800247 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:14.801265 (+ 1018us) server_negotiation.cc:407] Connection header received
0811 17:20:14.802732 (+ 1467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:14.802735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:14.802790 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:14.802875 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:14.804344 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:14.804922 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:14.806405 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:14.806773 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:14.806949 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:14.810314 (+ 3365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:20:14.810321 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:20:14.810327 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:20:14.810367 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:20:14.812114 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:20:14.812937 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:20:14.812944 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:20:14.812949 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:20:14.813049 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:20:14.813496 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:20:14.813502 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:20:14.813507 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:20:14.813796 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:20:14.814064 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 17:20:14.814676 (+ 612us) server_negotiation.cc:299] Negotiation successful
0811 17:20:14.815001 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 17:20:14.816139 26713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:14.800554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:37835 (local address 127.9.157.130:50889)
0811 17:20:14.801091 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:14.801139 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:14.802494 (+ 1355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:14.803078 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:14.803085 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:14.803495 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:14.804155 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:14.804170 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:14.805145 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:14.805148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:14.806247 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:14.806264 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:14.806979 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:14.806982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:14.807188 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:14.808020 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:14.808045 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:14.810100 (+ 2055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:20:14.812271 (+ 2171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:20:14.812282 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:20:14.812306 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:20:14.812774 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:20:14.813181 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:20:14.813184 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:20:14.813185 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:20:14.813378 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:20:14.813973 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:20:14.813981 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:20:14.814309 (+ 328us) client_negotiation.cc:769] Sending connection context
0811 17:20:14.814515 (+ 206us) client_negotiation.cc:240] Negotiation successful
0811 17:20:14.814863 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":124,"threads_started":1}
I20260811 17:20:14.817750 26711 heartbeater.cc:347] Connected to a master server at 127.9.157.190:37835
I20260811 17:20:14.818120 26711 heartbeater.cc:464] Registering TS with master...
I20260811 17:20:14.818933 26711 heartbeater.cc:511] Master 127.9.157.190:37835 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.157.131@KRBTEST.COM; defaulting to no policy
I20260811 17:20:14.821044 26225 ts_manager.cc:194] Registered new tserver with Master: 718afe47844b4f6dacee280d5cfbd117 (127.9.157.130:35231)
I20260811 17:20:14.821692 26225 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.130@KRBTEST.COM'} at 127.9.157.130:50889
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
I20260811 17:20:14.843202 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:0
--local_ip_for_outbound_sockets=127.9.157.131
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:37835
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:20:14.981725 26716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:20:14.982028 26716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:20:14.982074 26716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:20:14.986333 26716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:20:14.986418 26716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:20:14.986500 26716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:20:14.992172 26716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:37835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:20:14.993333 26716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:20:14.994367 26716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:20:15.005048 26723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:20:15.005139 26716 server_base.cc:1034] running on GCE node
W20260811 17:20:15.005100 26727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:20:15.005301 26725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:20:15.005738 26716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:20:15.006605 26716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:20:15.007773 26716 hybrid_clock.cc:648] HybridClock initialized: now 1786468815007743 us; error 39 us; skew 500 ppm
Aug 11 17:20:15 dist-test-slave-mjjr krb5kdc[26163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468815, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:20:15.010739 26716 init.cc:377] Logged in from keytab as kudu/127.9.157.131@KRBTEST.COM (short username kudu)
I20260811 17:20:15.011844 26716 webserver.cc:460] Webserver started at http://127.9.157.131:42673/ using document root <none> and password file <none>
I20260811 17:20:15.012447 26716 fs_manager.cc:359] Metadata directory not provided
I20260811 17:20:15.012511 26716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:20:15.012693 26716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:20:15.014837 26716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/data/instance:
uuid: "fe9cee2043614925a0dcbf24c4f015bc"
format_stamp: "Formatted at 2026-08-11 17:20:15 on dist-test-slave-mjjr"
server_key: "d38a28c92e9563ce2ae495fc3055df12"
server_key_iv: "e4b0a60c142df90d965dd794f3e40e94"
server_key_version: "encryptionkey@0"
I20260811 17:20:15.015390 26716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance:
uuid: "fe9cee2043614925a0dcbf24c4f015bc"
format_stamp: "Formatted at 2026-08-11 17:20:15 on dist-test-slave-mjjr"
server_key: "d38a28c92e9563ce2ae495fc3055df12"
server_key_iv: "e4b0a60c142df90d965dd794f3e40e94"
server_key_version: "encryptionkey@0"
I20260811 17:20:15.019361 26716 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 17:20:15.022323 26739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:15.023782 26716 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 17:20:15.023929 26716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "fe9cee2043614925a0dcbf24c4f015bc"
format_stamp: "Formatted at 2026-08-11 17:20:15 on dist-test-slave-mjjr"
server_key: "d38a28c92e9563ce2ae495fc3055df12"
server_key_iv: "e4b0a60c142df90d965dd794f3e40e94"
server_key_version: "encryptionkey@0"
I20260811 17:20:15.024058 26716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:20:15.045361 26716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:20:15.048732 26716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:20:15.049017 26716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:20:15.049662 26716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:20:15.050596 26716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:20:15.050645 26716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:15.050716 26716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:20:15.050750 26716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:20:15.061026 26716 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:33451
I20260811 17:20:15.061079 26904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:33451 every 8 connection(s)
I20260811 17:20:15.062120 26716 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:20:15.062741 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 26716
I20260811 17:20:15.062877 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance
I20260811 17:20:15.063238 9846 external_mini_cluster.cc:1428] Setting key f9a002e304bf49e400cebfd61a7ff538
Aug 11 17:20:15 dist-test-slave-mjjr krb5kdc[26163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468815, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:20:15.076994 26276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:15.064445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:32871 (local address 127.9.157.190:37835)
0811 17:20:15.064607 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:15.064612 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:15.065715 (+ 1103us) server_negotiation.cc:407] Connection header received
0811 17:20:15.067273 (+ 1558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:15.067276 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:15.067335 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:15.067448 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:15.068780 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:15.069314 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:15.070485 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:15.070710 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:15.070826 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:15.073393 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:20:15.073396 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:20:15.073398 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:20:15.073422 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:20:15.074694 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:20:15.075217 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:20:15.075220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:20:15.075222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:20:15.075305 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:20:15.075697 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:20:15.075700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:20:15.075704 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:20:15.075895 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:20:15.076011 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 17:20:15.076659 (+ 648us) server_negotiation.cc:299] Negotiation successful
0811 17:20:15.076798 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 17:20:15.078009 26907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:15.065009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:37835 (local address 127.9.157.131:32871)
0811 17:20:15.065566 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:15.065628 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:15.067047 (+ 1419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:15.067696 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:15.067706 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:15.068117 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:20:15.068632 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:15.068646 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:15.069457 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:15.069461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:15.070354 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:15.070364 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:15.070884 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:15.070888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:15.071065 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:15.071680 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:20:15.071708 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:20:15.073197 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:20:15.074816 (+ 1619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:20:15.074822 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:20:15.074839 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:20:15.075105 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:20:15.075411 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:20:15.075414 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:20:15.075416 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:20:15.075560 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:20:15.076009 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:20:15.076016 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:20:15.076340 (+ 324us) client_negotiation.cc:769] Sending connection context
0811 17:20:15.076588 (+ 248us) client_negotiation.cc:240] Negotiation successful
0811 17:20:15.076970 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":122,"threads_started":1}
I20260811 17:20:15.079145 26905 heartbeater.cc:347] Connected to a master server at 127.9.157.190:37835
I20260811 17:20:15.079427 26905 heartbeater.cc:464] Registering TS with master...
I20260811 17:20:15.080029 26905 heartbeater.cc:511] Master 127.9.157.190:37835 requested a full tablet report, sending...
I20260811 17:20:15.081179 26225 ts_manager.cc:194] Registered new tserver with Master: fe9cee2043614925a0dcbf24c4f015bc (127.9.157.131:33451)
I20260811 17:20:15.081741 26225 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.131@KRBTEST.COM'} at 127.9.157.131:32871
I20260811 17:20:15.089694 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:20:15.099713 26276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:15.091844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57576 (local address 127.9.157.190:37835)
0811 17:20:15.092023 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:15.092028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:15.092178 (+ 150us) server_negotiation.cc:407] Connection header received
0811 17:20:15.092348 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:15.092352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:15.092405 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:15.092491 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:20:15.093242 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:15.093776 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:15.094648 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:15.094894 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:15.095010 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:15.096484 (+ 1474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:20:15.096488 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:20:15.096491 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:20:15.096529 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:20:15.097802 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:20:15.098275 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:20:15.098279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:20:15.098281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:20:15.098372 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:20:15.098655 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:20:15.098659 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:20:15.098661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:20:15.098934 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:20:15.099092 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 17:20:15.099351 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 17:20:15.099478 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 17:20:15.103103 26225 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57576:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:20:15.106751 26225 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:20:15.122406 26926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:15.117237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54688 (local address 127.9.157.131:33451)
0811 17:20:15.117740 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:15.117751 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:15.117770 (+ 19us) server_negotiation.cc:407] Connection header received
0811 17:20:15.117842 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:15.117858 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:15.118028 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:15.118179 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:20:15.118407 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:15.119237 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:15.120812 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:15.121606 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:15.121724 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:15.121885 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 17:20:15.122113 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 17:20:15.122243 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":152,"threads_started":1}
I20260811 17:20:15.122395 26923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:15.116822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:33451 (local address 127.0.0.1:54688)
0811 17:20:15.117411 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:15.117427 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:15.117516 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:15.118167 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:15.118170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:15.118190 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:20:15.118281 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:15.118287 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:15.119383 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:15.119387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:15.120701 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:15.120712 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:15.121751 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:15.121754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:15.121880 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:15.121934 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 17:20:15.122062 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 17:20:15.122206 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":80,"threads_started":1}
I20260811 17:20:15.122813 26921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:15.116150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33790 (local address 127.9.157.130:35231)
0811 17:20:15.116623 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:15.116633 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:15.116654 (+ 21us) server_negotiation.cc:407] Connection header received
0811 17:20:15.116756 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:15.116761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:15.117426 (+ 665us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:15.117602 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:20:15.117927 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:15.119021 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:15.120729 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:15.121605 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:15.121724 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:15.121885 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 17:20:15.122597 (+ 712us) server_negotiation.cc:299] Negotiation successful
0811 17:20:15.122702 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":155,"threads_started":1}
I20260811 17:20:15.123062 26922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:15.116756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:42895 (local address 127.0.0.1:57366)
0811 17:20:15.117077 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:15.117091 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:15.117185 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:15.118362 (+ 1177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:15.118367 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:15.118387 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:20:15.118488 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:15.118494 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:15.119888 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:15.119891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:15.121017 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:15.121026 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:15.122076 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:15.122079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:15.122224 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:15.122244 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 17:20:15.122315 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 17:20:15.122387 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":29,"thread_start_us":149,"threads_started":1}
I20260811 17:20:15.122450 26920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:15.115990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:35231 (local address 127.0.0.1:33790)
0811 17:20:15.116362 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:15.116411 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:15.116539 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:15.117589 (+ 1050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:15.117595 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:15.117619 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:20:15.117793 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:15.117806 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:15.119168 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:15.119171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:15.120557 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:15.120569 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:15.121975 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:15.121984 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:15.122085 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:15.122164 (+ 79us) client_negotiation.cc:769] Sending connection context
0811 17:20:15.122235 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 17:20:15.122304 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":30,"thread_start_us":106,"threads_started":1}
I20260811 17:20:15.123389 26924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:15.116935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57366 (local address 127.9.157.129:42895)
0811 17:20:15.117734 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:15.117741 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:15.117767 (+ 26us) server_negotiation.cc:407] Connection header received
0811 17:20:15.117843 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:15.117847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:15.118028 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:15.118179 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:20:15.118623 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:15.119743 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:15.121140 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:15.121745 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:15.121860 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:15.122009 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 17:20:15.122501 (+ 492us) server_negotiation.cc:299] Negotiation successful
0811 17:20:15.123214 (+ 713us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":107,"threads_started":1}
I20260811 17:20:15.126030 26407 tablet_service.cc:1467] Processing CreateTablet for tablet 2545743edd474d38bcd4eca33a0645bb (DEFAULT_TABLE table=test-table [id=5b73dc61617d442cb951a381f808b502]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:20:15.126094 26814 tablet_service.cc:1467] Processing CreateTablet for tablet 2545743edd474d38bcd4eca33a0645bb (DEFAULT_TABLE table=test-table [id=5b73dc61617d442cb951a381f808b502]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:20:15.126094 26611 tablet_service.cc:1467] Processing CreateTablet for tablet 2545743edd474d38bcd4eca33a0645bb (DEFAULT_TABLE table=test-table [id=5b73dc61617d442cb951a381f808b502]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:20:15.127249 26814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2545743edd474d38bcd4eca33a0645bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:20:15.127235 26407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2545743edd474d38bcd4eca33a0645bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:20:15.127243 26611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2545743edd474d38bcd4eca33a0645bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:20:15.134279 26927 tablet_bootstrap.cc:492] T 2545743edd474d38bcd4eca33a0645bb P fed7ff6acbac4e5ca9e5da9ae0a8e173: Bootstrap starting.
I20260811 17:20:15.135609 26929 tablet_bootstrap.cc:492] T 2545743edd474d38bcd4eca33a0645bb P fe9cee2043614925a0dcbf24c4f015bc: Bootstrap starting.
I20260811 17:20:15.136009 26930 tablet_bootstrap.cc:492] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117: Bootstrap starting.
I20260811 17:20:15.136834 26927 tablet_bootstrap.cc:654] T 2545743edd474d38bcd4eca33a0645bb P fed7ff6acbac4e5ca9e5da9ae0a8e173: Neither blocks nor log segments found. Creating new log.
I20260811 17:20:15.137630 26927 log.cc:824] T 2545743edd474d38bcd4eca33a0645bb P fed7ff6acbac4e5ca9e5da9ae0a8e173: Log is configured to *not* fsync() on all Append() calls
I20260811 17:20:15.138450 26930 tablet_bootstrap.cc:654] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117: Neither blocks nor log segments found. Creating new log.
I20260811 17:20:15.138451 26929 tablet_bootstrap.cc:654] T 2545743edd474d38bcd4eca33a0645bb P fe9cee2043614925a0dcbf24c4f015bc: Neither blocks nor log segments found. Creating new log.
I20260811 17:20:15.139359 26930 log.cc:824] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117: Log is configured to *not* fsync() on all Append() calls
I20260811 17:20:15.139369 26929 log.cc:824] T 2545743edd474d38bcd4eca33a0645bb P fe9cee2043614925a0dcbf24c4f015bc: Log is configured to *not* fsync() on all Append() calls
I20260811 17:20:15.140281 26927 tablet_bootstrap.cc:492] T 2545743edd474d38bcd4eca33a0645bb P fed7ff6acbac4e5ca9e5da9ae0a8e173: No bootstrap required, opened a new log
I20260811 17:20:15.140516 26927 ts_tablet_manager.cc:1397] T 2545743edd474d38bcd4eca33a0645bb P fed7ff6acbac4e5ca9e5da9ae0a8e173: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 17:20:15.141390 26930 tablet_bootstrap.cc:492] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117: No bootstrap required, opened a new log
I20260811 17:20:15.141551 26929 tablet_bootstrap.cc:492] T 2545743edd474d38bcd4eca33a0645bb P fe9cee2043614925a0dcbf24c4f015bc: No bootstrap required, opened a new log
I20260811 17:20:15.141628 26930 ts_tablet_manager.cc:1397] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 17:20:15.141752 26929 ts_tablet_manager.cc:1397] T 2545743edd474d38bcd4eca33a0645bb P fe9cee2043614925a0dcbf24c4f015bc: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260811 17:20:15.144791 26930 raft_consensus.cc:348] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718afe47844b4f6dacee280d5cfbd117" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35231 } } peers { permanent_uuid: "fed7ff6acbac4e5ca9e5da9ae0a8e173" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 42895 } } peers { permanent_uuid: "fe9cee2043614925a0dcbf24c4f015bc" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 33451 } }
I20260811 17:20:15.145072 26930 raft_consensus.cc:374] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:20:15.145128 26930 raft_consensus.cc:729] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 718afe47844b4f6dacee280d5cfbd117, State: Initialized, Role: FOLLOWER
I20260811 17:20:15.145372 26927 raft_consensus.cc:348] T 2545743edd474d38bcd4eca33a0645bb P fed7ff6acbac4e5ca9e5da9ae0a8e173 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718afe47844b4f6dacee280d5cfbd117" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35231 } } peers { permanent_uuid: "fed7ff6acbac4e5ca9e5da9ae0a8e173" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 42895 } } peers { permanent_uuid: "fe9cee2043614925a0dcbf24c4f015bc" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 33451 } }
I20260811 17:20:15.145650 26927 raft_consensus.cc:374] T 2545743edd474d38bcd4eca33a0645bb P fed7ff6acbac4e5ca9e5da9ae0a8e173 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:20:15.145713 26927 raft_consensus.cc:729] T 2545743edd474d38bcd4eca33a0645bb P fed7ff6acbac4e5ca9e5da9ae0a8e173 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fed7ff6acbac4e5ca9e5da9ae0a8e173, State: Initialized, Role: FOLLOWER
I20260811 17:20:15.145637 26930 consensus_queue.cc:260] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718afe47844b4f6dacee280d5cfbd117" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35231 } } peers { permanent_uuid: "fed7ff6acbac4e5ca9e5da9ae0a8e173" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 42895 } } peers { permanent_uuid: "fe9cee2043614925a0dcbf24c4f015bc" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 33451 } }
I20260811 17:20:15.145771 26929 raft_consensus.cc:348] T 2545743edd474d38bcd4eca33a0645bb P fe9cee2043614925a0dcbf24c4f015bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718afe47844b4f6dacee280d5cfbd117" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35231 } } peers { permanent_uuid: "fed7ff6acbac4e5ca9e5da9ae0a8e173" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 42895 } } peers { permanent_uuid: "fe9cee2043614925a0dcbf24c4f015bc" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 33451 } }
I20260811 17:20:15.146041 26929 raft_consensus.cc:374] T 2545743edd474d38bcd4eca33a0645bb P fe9cee2043614925a0dcbf24c4f015bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:20:15.146108 26929 raft_consensus.cc:729] T 2545743edd474d38bcd4eca33a0645bb P fe9cee2043614925a0dcbf24c4f015bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe9cee2043614925a0dcbf24c4f015bc, State: Initialized, Role: FOLLOWER
I20260811 17:20:15.146539 26711 heartbeater.cc:503] Master 127.9.157.190:37835 was elected leader, sending a full tablet report...
I20260811 17:20:15.146632 26929 consensus_queue.cc:260] T 2545743edd474d38bcd4eca33a0645bb P fe9cee2043614925a0dcbf24c4f015bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718afe47844b4f6dacee280d5cfbd117" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35231 } } peers { permanent_uuid: "fed7ff6acbac4e5ca9e5da9ae0a8e173" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 42895 } } peers { permanent_uuid: "fe9cee2043614925a0dcbf24c4f015bc" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 33451 } }
I20260811 17:20:15.146847 26930 ts_tablet_manager.cc:1428] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 17:20:15.147418 26927 consensus_queue.cc:260] T 2545743edd474d38bcd4eca33a0645bb P fed7ff6acbac4e5ca9e5da9ae0a8e173 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718afe47844b4f6dacee280d5cfbd117" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35231 } } peers { permanent_uuid: "fed7ff6acbac4e5ca9e5da9ae0a8e173" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 42895 } } peers { permanent_uuid: "fe9cee2043614925a0dcbf24c4f015bc" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 33451 } }
I20260811 17:20:15.147599 26905 heartbeater.cc:503] Master 127.9.157.190:37835 was elected leader, sending a full tablet report...
I20260811 17:20:15.147691 26929 ts_tablet_manager.cc:1428] T 2545743edd474d38bcd4eca33a0645bb P fe9cee2043614925a0dcbf24c4f015bc: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260811 17:20:15.148308 26509 heartbeater.cc:503] Master 127.9.157.190:37835 was elected leader, sending a full tablet report...
I20260811 17:20:15.148448 26927 ts_tablet_manager.cc:1428] T 2545743edd474d38bcd4eca33a0645bb P fed7ff6acbac4e5ca9e5da9ae0a8e173: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260811 17:20:15.223135 26934 raft_consensus.cc:482] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:20:15.223413 26934 raft_consensus.cc:504] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718afe47844b4f6dacee280d5cfbd117" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35231 } } peers { permanent_uuid: "fed7ff6acbac4e5ca9e5da9ae0a8e173" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 42895 } } peers { permanent_uuid: "fe9cee2043614925a0dcbf24c4f015bc" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 33451 } }
I20260811 17:20:15.224787 26934 leader_election.cc:288] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fed7ff6acbac4e5ca9e5da9ae0a8e173 (127.9.157.129:42895), fe9cee2043614925a0dcbf24c4f015bc (127.9.157.131:33451)
I20260811 17:20:15.228663 26713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:15.225041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:42895 (local address 127.9.157.130:48709)
0811 17:20:15.225169 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:15.225190 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:15.225300 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:15.225631 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:15.225635 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:15.225654 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:20:15.225786 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:15.225793 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:15.226571 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:15.226577 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:15.227433 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:15.227467 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:15.228214 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:15.228218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:15.228338 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:15.228363 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 17:20:15.228445 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 17:20:15.228524 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 17:20:15.228664 26924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:15.225182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:48709 (local address 127.9.157.129:42895)
0811 17:20:15.225354 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:15.225359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:15.225380 (+ 21us) server_negotiation.cc:407] Connection header received
0811 17:20:15.225443 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:15.225449 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:15.225508 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:15.225603 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:20:15.225912 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:15.226433 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:15.227595 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:15.228069 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:15.228219 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:15.228289 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 17:20:15.228462 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 17:20:15.228537 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 17:20:15.229195 26938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:15.225160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:33451 (local address 127.9.157.130:59943)
0811 17:20:15.225586 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:15.225597 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 17:20:15.225714 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:20:15.225988 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:20:15.225991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:20:15.226006 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:20:15.226129 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:15.226134 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:15.226848 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:15.226851 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:15.227871 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:20:15.227881 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:20:15.228740 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:20:15.228743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:20:15.228845 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:15.228866 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 17:20:15.228963 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 17:20:15.229025 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":146,"threads_started":1}
I20260811 17:20:15.229225 26926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:15.225375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:59943 (local address 127.9.157.131:33451)
0811 17:20:15.225508 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:15.225513 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:15.225678 (+ 165us) server_negotiation.cc:407] Connection header received
0811 17:20:15.225834 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:15.225837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:15.225883 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:15.225976 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:20:15.226236 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:15.226735 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:15.227992 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:15.228602 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:15.228739 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:15.228788 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 17:20:15.229007 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 17:20:15.229090 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 17:20:15.229530 26437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2545743edd474d38bcd4eca33a0645bb" candidate_uuid: "718afe47844b4f6dacee280d5cfbd117" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fed7ff6acbac4e5ca9e5da9ae0a8e173" is_pre_election: true
I20260811 17:20:15.229903 26437 raft_consensus.cc:2454] T 2545743edd474d38bcd4eca33a0645bb P fed7ff6acbac4e5ca9e5da9ae0a8e173 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 718afe47844b4f6dacee280d5cfbd117 in term 0.
I20260811 17:20:15.230050 26847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2545743edd474d38bcd4eca33a0645bb" candidate_uuid: "718afe47844b4f6dacee280d5cfbd117" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe9cee2043614925a0dcbf24c4f015bc" is_pre_election: true
I20260811 17:20:15.230361 26847 raft_consensus.cc:2454] T 2545743edd474d38bcd4eca33a0645bb P fe9cee2043614925a0dcbf24c4f015bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 718afe47844b4f6dacee280d5cfbd117 in term 0.
I20260811 17:20:15.230492 26549 leader_election.cc:302] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 718afe47844b4f6dacee280d5cfbd117, fed7ff6acbac4e5ca9e5da9ae0a8e173; no voters:
I20260811 17:20:15.230831 26934 raft_consensus.cc:2781] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:20:15.230928 26934 raft_consensus.cc:482] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:20:15.230973 26934 raft_consensus.cc:3037] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:20:15.232326 26934 raft_consensus.cc:504] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718afe47844b4f6dacee280d5cfbd117" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35231 } } peers { permanent_uuid: "fed7ff6acbac4e5ca9e5da9ae0a8e173" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 42895 } } peers { permanent_uuid: "fe9cee2043614925a0dcbf24c4f015bc" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 33451 } }
I20260811 17:20:15.232784 26934 leader_election.cc:288] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117 [CANDIDATE]: Term 1 election: Requested vote from peers fed7ff6acbac4e5ca9e5da9ae0a8e173 (127.9.157.129:42895), fe9cee2043614925a0dcbf24c4f015bc (127.9.157.131:33451)
I20260811 17:20:15.233233 26847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2545743edd474d38bcd4eca33a0645bb" candidate_uuid: "718afe47844b4f6dacee280d5cfbd117" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe9cee2043614925a0dcbf24c4f015bc"
I20260811 17:20:15.233338 26437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2545743edd474d38bcd4eca33a0645bb" candidate_uuid: "718afe47844b4f6dacee280d5cfbd117" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fed7ff6acbac4e5ca9e5da9ae0a8e173"
I20260811 17:20:15.233407 26847 raft_consensus.cc:3037] T 2545743edd474d38bcd4eca33a0645bb P fe9cee2043614925a0dcbf24c4f015bc [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:20:15.233489 26437 raft_consensus.cc:3037] T 2545743edd474d38bcd4eca33a0645bb P fed7ff6acbac4e5ca9e5da9ae0a8e173 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:20:15.234898 26847 raft_consensus.cc:2454] T 2545743edd474d38bcd4eca33a0645bb P fe9cee2043614925a0dcbf24c4f015bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 718afe47844b4f6dacee280d5cfbd117 in term 1.
I20260811 17:20:15.234898 26437 raft_consensus.cc:2454] T 2545743edd474d38bcd4eca33a0645bb P fed7ff6acbac4e5ca9e5da9ae0a8e173 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 718afe47844b4f6dacee280d5cfbd117 in term 1.
I20260811 17:20:15.235281 26549 leader_election.cc:302] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 718afe47844b4f6dacee280d5cfbd117, fed7ff6acbac4e5ca9e5da9ae0a8e173; no voters:
I20260811 17:20:15.235697 26934 raft_consensus.cc:2781] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:20:15.235993 26934 raft_consensus.cc:686] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117 [term 1 LEADER]: Becoming Leader. State: Replica: 718afe47844b4f6dacee280d5cfbd117, State: Running, Role: LEADER
I20260811 17:20:15.236369 26934 consensus_queue.cc:237] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718afe47844b4f6dacee280d5cfbd117" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35231 } } peers { permanent_uuid: "fed7ff6acbac4e5ca9e5da9ae0a8e173" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 42895 } } peers { permanent_uuid: "fe9cee2043614925a0dcbf24c4f015bc" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 33451 } }
I20260811 17:20:15.239984 26225 catalog_manager.cc:5495] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117 reported cstate change: term changed from 0 to 1, leader changed from <none> to 718afe47844b4f6dacee280d5cfbd117 (127.9.157.130). New cstate: current_term: 1 leader_uuid: "718afe47844b4f6dacee280d5cfbd117" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718afe47844b4f6dacee280d5cfbd117" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fed7ff6acbac4e5ca9e5da9ae0a8e173" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 42895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe9cee2043614925a0dcbf24c4f015bc" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 33451 } health_report { overall_health: UNKNOWN } } }
I20260811 17:20:15.277942 26921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:15.274112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33792 (local address 127.9.157.130:35231)
0811 17:20:15.274272 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 17:20:15.274277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:20:15.274294 (+ 17us) server_negotiation.cc:407] Connection header received
0811 17:20:15.274442 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:20:15.274445 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:20:15.274508 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:20:15.274634 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:20:15.274903 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:15.275408 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:15.276331 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:20:15.276560 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:20:15.276678 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:20:15.277027 (+ 349us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:20:15.277509 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:20:15.277654 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 17:20:15.277696 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 17:20:15.277764 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 17:20:15.284773 26437 raft_consensus.cc:1261] T 2545743edd474d38bcd4eca33a0645bb P fed7ff6acbac4e5ca9e5da9ae0a8e173 [term 1 FOLLOWER]: Refusing update from remote peer 718afe47844b4f6dacee280d5cfbd117: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:20:15.284773 26847 raft_consensus.cc:1261] T 2545743edd474d38bcd4eca33a0645bb P fe9cee2043614925a0dcbf24c4f015bc [term 1 FOLLOWER]: Refusing update from remote peer 718afe47844b4f6dacee280d5cfbd117: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:20:15.285607 26934 consensus_queue.cc:1035] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe9cee2043614925a0dcbf24c4f015bc" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 33451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:20:15.285837 26940 consensus_queue.cc:1035] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fed7ff6acbac4e5ca9e5da9ae0a8e173" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 42895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:20:15.293562 26943 mvcc.cc:205] Tried to move back new op lower bound from 7317376267397820416 to 7317376267212066816. Current Snapshot: MvccSnapshot[applied={T|T < 7317376267397820416}]
I20260811 17:20:15.293629 26945 mvcc.cc:205] Tried to move back new op lower bound from 7317376267397820416 to 7317376267212066816. Current Snapshot: MvccSnapshot[applied={T|T < 7317376267397820416}]
I20260811 17:20:15.295748 26944 mvcc.cc:205] Tried to move back new op lower bound from 7317376267397820416 to 7317376267212066816. Current Snapshot: MvccSnapshot[applied={T|T < 7317376267397820416}]
I20260811 17:20:15.303568 26224 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:20:15.303841 26224 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:20:15.306918 26224 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 82003876dc9046f49f786a1a961d9e81: Sending DeleteTablet for 3 replicas of tablet 2545743edd474d38bcd4eca33a0645bb
I20260811 17:20:15.308040 26814 tablet_service.cc:1514] Processing DeleteTablet for tablet 2545743edd474d38bcd4eca33a0645bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:20:15 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:54688
I20260811 17:20:15.308147 26407 tablet_service.cc:1514] Processing DeleteTablet for tablet 2545743edd474d38bcd4eca33a0645bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:20:15 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:57366
I20260811 17:20:15.308755 26958 tablet_replica.cc:330] T 2545743edd474d38bcd4eca33a0645bb P fed7ff6acbac4e5ca9e5da9ae0a8e173: stopping tablet replica
I20260811 17:20:15.308796 26957 tablet_replica.cc:330] T 2545743edd474d38bcd4eca33a0645bb P fe9cee2043614925a0dcbf24c4f015bc: stopping tablet replica
I20260811 17:20:15.308796 26611 tablet_service.cc:1514] Processing DeleteTablet for tablet 2545743edd474d38bcd4eca33a0645bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:20:15 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:33790
I20260811 17:20:15.309175 26958 raft_consensus.cc:2229] T 2545743edd474d38bcd4eca33a0645bb P fed7ff6acbac4e5ca9e5da9ae0a8e173 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:20:15.309470 26957 raft_consensus.cc:2229] T 2545743edd474d38bcd4eca33a0645bb P fe9cee2043614925a0dcbf24c4f015bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:20:15.309612 26958 raft_consensus.cc:2258] T 2545743edd474d38bcd4eca33a0645bb P fed7ff6acbac4e5ca9e5da9ae0a8e173 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:20:15.310077 26957 raft_consensus.cc:2258] T 2545743edd474d38bcd4eca33a0645bb P fe9cee2043614925a0dcbf24c4f015bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:20:15.310570 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 26312
I20260811 17:20:15.315501 26957 ts_tablet_manager.cc:1905] T 2545743edd474d38bcd4eca33a0645bb P fe9cee2043614925a0dcbf24c4f015bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:20:15.319743 26957 ts_tablet_manager.cc:1918] T 2545743edd474d38bcd4eca33a0645bb P fe9cee2043614925a0dcbf24c4f015bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:20:15.320071 26957 log.cc:1196] T 2545743edd474d38bcd4eca33a0645bb P fe9cee2043614925a0dcbf24c4f015bc: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-2/wal/wals/2545743edd474d38bcd4eca33a0645bb
I20260811 17:20:15.320511 26957 ts_tablet_manager.cc:1939] T 2545743edd474d38bcd4eca33a0645bb P fe9cee2043614925a0dcbf24c4f015bc: Deleting consensus metadata
I20260811 17:20:15.322871 26201 catalog_manager.cc:4841] TS fe9cee2043614925a0dcbf24c4f015bc (127.9.157.131:33451): tablet 2545743edd474d38bcd4eca33a0645bb (table test-table [id=5b73dc61617d442cb951a381f808b502]) successfully deleted
I20260811 17:20:15.325161 26961 tablet_replica.cc:330] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117: stopping tablet replica
I20260811 17:20:15.325714 26961 raft_consensus.cc:2229] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117 [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:20:15.326409 26961 raft_consensus.cc:2258] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:20:15.328766 26961 ts_tablet_manager.cc:1905] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:20:15.332991 26961 ts_tablet_manager.cc:1918] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:20:15.333199 26961 log.cc:1196] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786468743878666-9846-0/minicluster-data/ts-1/wal/wals/2545743edd474d38bcd4eca33a0645bb
I20260811 17:20:15.333585 26961 ts_tablet_manager.cc:1939] T 2545743edd474d38bcd4eca33a0645bb P 718afe47844b4f6dacee280d5cfbd117: Deleting consensus metadata
I20260811 17:20:15.335258 26206 catalog_manager.cc:4841] TS 718afe47844b4f6dacee280d5cfbd117 (127.9.157.130:35231): tablet 2545743edd474d38bcd4eca33a0645bb (table test-table [id=5b73dc61617d442cb951a381f808b502]) successfully deleted
W20260811 17:20:15.336279 26206 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.157.129:42895 (error 108)
I20260811 17:20:15.336395 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 26518
I20260811 17:20:15.343466 26920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:20:15.343114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:42895 (local address 127.0.0.1:57370)
0811 17:20:15.343274 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 17:20:15.343372 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.129:42895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 17:20:15.343866 26206 catalog_manager.cc:4584] TS fed7ff6acbac4e5ca9e5da9ae0a8e173 (127.9.157.129:42895): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2545743edd474d38bcd4eca33a0645bb: Network error: Client connection negotiation failed: client connection to 127.9.157.129:42895: connect: Connection refused (error 111)
I20260811 17:20:15.349045 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 26716
I20260811 17:20:15.369530 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 26179
2026-08-11T17:20:15Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2759 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:20:15 dist-test-slave-mjjr krb5kdc[26969](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:20:15 dist-test-slave-mjjr krb5kdc[26969](info): set up 2 sockets
Aug 11 17:20:15 dist-test-slave-mjjr krb5kdc[26969](info): commencing operation
krb5kdc: starting...
W20260811 17:20:16.699065 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.297s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:20:16 dist-test-slave-mjjr krb5kdc[26969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:20:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:20:16Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.157.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.157.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.157.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/rangeradmin_127.9.157.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.157.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/rangeradmin_127.9.157.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.157.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.157.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.157.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/rangerlookup_127.9.157.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.157.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/rangerlookup_127.9.157.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.157.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.157.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.157.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/HTTP_127.9.157.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.157.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/HTTP_127.9.157.148@KRBTEST.COM.keytab.
I20260811 17:20:16.890862 9846 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasklEMhqg/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/postgres -l logfile start
2026-08-11 17:20:19.474 UTC [27010] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 17:20:19.474 UTC [27010] LOG: listening on IPv4 address "127.9.157.148", port 37143
2026-08-11 17:20:19.478 UTC [27010] LOG: listening on Unix socket "/tmp/.s.PGSQL.37143"
2026-08-11 17:20:19.501 UTC [27014] LOG: database system was shut down at 2026-08-11 17:20:18 UTC
2026-08-11 17:20:19.505 UTC [27010] LOG: database system is ready to accept connections
I20260811 17:20:20.679082 9846 mini_postgres.cc:96] Postgres bound to 37143
2026-08-11 17:20:20.688 UTC [27025] FATAL: database "slave" does not exist
127.9.157.148:37143 - accepting connections
I20260811 17:20:20.688997 9846 mini_ranger.cc:161] Starting Ranger...
I20260811 17:20:20.708135 9846 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 17:20:21.221154 9846 mini_ranger.cc:87] Created ranger Postgres database
I20260811 17:20:21.221271 9846 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ranger-admin
2026-08-11 17:20:21,610 [I] DB FLAVOR :POSTGRES
2026-08-11 17:20:21,611 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 17:20:21,611 [I] Checking connection..
2026-08-11 17:20:21,611 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:37143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 17:20:21,922 [I] Checking connection passed.
2026-08-11 17:20:21,923 [I] --------- Verifying version history table ---------
2026-08-11 17:20:21,923 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:37143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 17:20:22,208 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 17:20:22,208 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:37143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 17:20:22,495 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 17:20:22,496 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 17:20:22,496 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:37143/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 17:20:22.761 UTC [27117] WARNING: there is no transaction in progress
2026-08-11 17:20:22,768 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 17:20:22,769 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:37143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 17:20:23,045 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 17:20:23,046 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 17:20:23,046 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:37143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 17:20:23,326 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:37143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 17:20:23,574 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:37143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-mjjr.c.gcp-upstream.internal','N') ;"
2026-08-11 17:20:23,848 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:37143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 17:20:24,123 [I] Table x_portal_user does not exist in database ranger
2026-08-11 17:20:24,124 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:37143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 17:20:24,380 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 17:20:24,381 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:37143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 17:20:24,638 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:37143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 17:20:24,929 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 17:20:24,930 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:37143/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 17:20:25.995 UTC [27269] WARNING: there is no transaction in progress
2026-08-11 17:20:26.013 UTC [27269] WARNING: there is no transaction in progress
2026-08-11 17:20:26.030 UTC [27269] WARNING: there is no transaction in progress
2026-08-11 17:20:26.048 UTC [27269] WARNING: there is no transaction in progress
2026-08-11 17:20:26.066 UTC [27269] WARNING: there is no transaction in progress
2026-08-11 17:20:26.085 UTC [27269] WARNING: there is no transaction in progress
2026-08-11 17:20:26.154 UTC [27269] WARNING: there is no transaction in progress
2026-08-11 17:20:26.162 UTC [27269] WARNING: there is no transaction in progress
2026-08-11 17:20:26.178 UTC [27269] WARNING: there is no transaction in progress
2026-08-11 17:20:26.189 UTC [27269] WARNING: there is no transaction in progress
2026-08-11 17:20:26.200 UTC [27269] WARNING: there is no transaction in progress
2026-08-11 17:20:26.215 UTC [27269] WARNING: there is no transaction in progress
2026-08-11 17:20:26.226 UTC [27269] WARNING: there is no transaction in progress
2026-08-11 17:20:26.234 UTC [27269] WARNING: there is no transaction in progress
2026-08-11 17:20:26.243 UTC [27269] WARNING: there is no transaction in progress
2026-08-11 17:20:26.251 UTC [27269] WARNING: there is no transaction in progress
2026-08-11 17:20:27,108 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 17:20:27,109 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:37143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 17:20:27,537 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 17:20:27,538 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:37143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 17:20:27,919 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 17:20:27,920 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:37143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 17:20:28,231 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 17:20:28,232 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:37143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 17:20:28,480 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:37143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 17:20:28,751 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:37143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-mjjr.c.gcp-upstream.internal';"
2026-08-11 17:20:28,997 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 17:20:28,997 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:37143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 17:20:29,277 [I] DB_PATCHES have already been applied
I20260811 17:20:29.283378 9846 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ranger-admin:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasklEMhqg/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 17:20:29.283461 9846 mini_ranger.cc:193] Using host: 127.9.157.148
I20260811 17:20:29.289496 9846 mini_ranger.cc:227] Ranger admin URL: http://127.9.157.148:44933
Aug 11, 2026 5:20:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 5:20:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 5:20:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 5:20:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 5:20:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 5:20:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 5:20:30 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 17:20:30.709223 9846 mini_ranger.cc:160] Time spent starting Ranger: real 10.020s user 0.007s sys 0.007s
Aug 11, 2026 5:20:52 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 5:20:52 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 11, 2026 5:20:52 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 5:20:52 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 5:20:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 5:20:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 5:20:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 5:20:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 5:20:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 5:20:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 5:20:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 5:20:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 5:20:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 5:20:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 5:20:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 5:20:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 5:20:52 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 5:20:53 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 17:20:54.105425 9846 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.157.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.157.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.157.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/rangerkms_127.9.157.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.157.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/rangerkms_127.9.157.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.157.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.157.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.157.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/HTTP_127.9.157.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.157.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/HTTP_127.9.157.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 17:20:54 dist-test-slave-mjjr krb5kdc[26969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468854, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 17:20:54.206453 9846 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 17:20:54.220542 9846 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 17:20:55.125643 9846 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 17:20:55.125761 9846 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ranger-kms
I20260811 17:20:55.125880 9846 mini_ranger_kms.cc:227] Using postgres at 127.9.157.148:37143
2026-08-11 17:20:55,396 [I] DB FLAVOR :POSTGRES
2026-08-11 17:20:55,397 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 17:20:55,397 [I] Checking connection
2026-08-11 17:20:55,397 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:37143/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 17:20:55,728 [I] connection success
2026-08-11 17:20:55,728 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 17:20:55,728 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 17:20:55,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:37143/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 17:20:56,002 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 17:20:56,002 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 17:20:56,003 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 17:20:56,003 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:37143/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 17:20:56,316 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 17:20:56.858816 9846 mini_ranger_kms.cc:310] Created kms service
I20260811 17:20:57.114904 9846 mini_ranger_kms.cc:326] Created kudu user
I20260811 17:20:57.185926 9846 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 17:20:57 dist-test-slave-mjjr krb5kdc[26969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468854, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.157.148@KRBTEST.COM
I20260811 17:20:57.701201 9846 mini_ranger_kms.cc:384] Added ranger policy
I20260811 17:20:57.701390 9846 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ranger-kms:/tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 17:20:57.701413 9846 mini_ranger_kms.cc:240] Using host: 127.9.157.148
I20260811 17:20:57.706780 9846 mini_ranger_kms.cc:276] Ranger KMS PID: 27526
I20260811 17:20:57.706967 9846 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.157.148:56779
Aug 11, 2026 5:20:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 5:20:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 5:20:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 5:20:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 5:20:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 5:20:59 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 17:21:00 dist-test-slave-mjjr krb5kdc[26969](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786468860, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.157.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:21:00.516387 9846 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.310s user 0.014s sys 0.243s
I20260811 17:21:00.516546 9846 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 17:21:00.516678 9846 mini_ranger_kms.cc:401] 127.9.157.148:56779/kms/v1/keys
Aug 11 17:21:00 dist-test-slave-mjjr krb5kdc[26969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468860, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.157.148@KRBTEST.COM for HTTP/127.9.157.148@KRBTEST.COM
Aug 11 17:21:01 dist-test-slave-mjjr krb5kdc[26969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468860, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.157.148@KRBTEST.COM for HTTP/127.9.157.148@KRBTEST.COM
Aug 11 17:21:03 dist-test-slave-mjjr krb5kdc[26969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:21:03.843415 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:36313
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:46477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:36313
--ranger_config_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.157.148:56779/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:21:03.991813 27600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:21:03.992143 27600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:21:03.992188 27600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:21:03.996479 27600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:21:03.996567 27600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:21:03.996587 27600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:21:03.996603 27600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:21:03.996619 27600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:21:04.002683 27600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:46477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.157.148:56779/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:36313
--ranger_config_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:36313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:21:04.004035 27600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:21:04.005211 27600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:21:04.015360 27607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:21:04.015470 27600 server_base.cc:1034] running on GCE node
W20260811 17:21:04.015413 27609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:21:04.015726 27611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:21:04.016374 27600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:21:04.017655 27600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:21:04.018893 27600 hybrid_clock.cc:648] HybridClock initialized: now 1786468864018870 us; error 31 us; skew 500 ppm
Aug 11 17:21:04 dist-test-slave-mjjr krb5kdc[26969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468864, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:21:04.026981 27600 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:21:04.028370 27600 webserver.cc:460] Webserver started at http://127.9.157.190:38895/ using document root <none> and password file <none>
I20260811 17:21:04.029073 27600 fs_manager.cc:359] Metadata directory not provided
I20260811 17:21:04.029181 27600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:21:04.029536 27600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 17:21:04 dist-test-slave-mjjr krb5kdc[26969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468864, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for HTTP/127.9.157.148@KRBTEST.COM
I20260811 17:21:04.093525 27600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "75c7c6919f434684b19eadeb15174ea9"
format_stamp: "Formatted at 2026-08-11 17:21:04 on dist-test-slave-mjjr"
server_key: "58ec03b788e5cac0906454ff3003cb93"
server_key_iv: "a4d7df61f928a285e5f8e32291f18cd0"
server_key_version: "kuduclusterkey@0"
I20260811 17:21:04.094415 27600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "75c7c6919f434684b19eadeb15174ea9"
format_stamp: "Formatted at 2026-08-11 17:21:04 on dist-test-slave-mjjr"
server_key: "58ec03b788e5cac0906454ff3003cb93"
server_key_iv: "a4d7df61f928a285e5f8e32291f18cd0"
server_key_version: "kuduclusterkey@0"
I20260811 17:21:04.100383 27600 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 17:21:04.118239 27626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:21:04.120195 27600 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 17:21:04.120441 27600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "75c7c6919f434684b19eadeb15174ea9"
format_stamp: "Formatted at 2026-08-11 17:21:04 on dist-test-slave-mjjr"
server_key: "58ec03b788e5cac0906454ff3003cb93"
server_key_iv: "a4d7df61f928a285e5f8e32291f18cd0"
server_key_version: "kuduclusterkey@0"
I20260811 17:21:04.120621 27600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:21:04.164119 27600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:21:04.167912 27600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:21:04.168180 27600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:21:04.177758 27600 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:36313
I20260811 17:21:04.177796 27704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:36313 every 8 connection(s)
I20260811 17:21:04.179003 27600 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:21:04.182530 27706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:21:04.186755 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 27600
I20260811 17:21:04.186900 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:21:04.187232 9846 external_mini_cluster.cc:1428] Setting key 72c6299da2cfe0eaba4e7ed51a29e1b9
I20260811 17:21:04.189728 27706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75c7c6919f434684b19eadeb15174ea9: Bootstrap starting.
I20260811 17:21:04.192521 27706 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75c7c6919f434684b19eadeb15174ea9: Neither blocks nor log segments found. Creating new log.
I20260811 17:21:04.193426 27706 log.cc:824] T 00000000000000000000000000000000 P 75c7c6919f434684b19eadeb15174ea9: Log is configured to *not* fsync() on all Append() calls
I20260811 17:21:04.195814 27706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75c7c6919f434684b19eadeb15174ea9: No bootstrap required, opened a new log
Aug 11 17:21:04 dist-test-slave-mjjr krb5kdc[26969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:21:04.200304 27706 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75c7c6919f434684b19eadeb15174ea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c7c6919f434684b19eadeb15174ea9" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 36313 } }
I20260811 17:21:04.200640 27706 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75c7c6919f434684b19eadeb15174ea9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:21:04.200709 27706 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75c7c6919f434684b19eadeb15174ea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75c7c6919f434684b19eadeb15174ea9, State: Initialized, Role: FOLLOWER
I20260811 17:21:04.201274 27706 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75c7c6919f434684b19eadeb15174ea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c7c6919f434684b19eadeb15174ea9" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 36313 } }
I20260811 17:21:04.201454 27706 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75c7c6919f434684b19eadeb15174ea9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:21:04.201548 27706 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75c7c6919f434684b19eadeb15174ea9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:21:04.201685 27706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75c7c6919f434684b19eadeb15174ea9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:21:04.202885 27706 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75c7c6919f434684b19eadeb15174ea9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c7c6919f434684b19eadeb15174ea9" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 36313 } }
I20260811 17:21:04.203071 27710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:04.189142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35762 (local address 127.9.157.190:36313)
0811 17:21:04.189703 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0811 17:21:04.189721 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 17:21:04.189765 (+ 44us) server_negotiation.cc:407] Connection header received
0811 17:21:04.190562 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:21:04.190583 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:21:04.190969 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:21:04.191378 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:21:04.192332 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:04.193456 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:04.194420 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:04.194942 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:04.195139 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:04.198014 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:21:04.198020 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:21:04.198040 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:21:04.198073 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:21:04.199907 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:21:04.200424 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:21:04.200430 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:21:04.200435 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:21:04.200557 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:21:04.200881 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:21:04.200885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:21:04.200887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:21:04.201366 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:21:04.201628 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 17:21:04.201894 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 17:21:04.202217 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":168,"threads_started":1}
I20260811 17:21:04.203361 27706 leader_election.cc:302] T 00000000000000000000000000000000 P 75c7c6919f434684b19eadeb15174ea9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75c7c6919f434684b19eadeb15174ea9; no voters:
I20260811 17:21:04.203831 27706 leader_election.cc:288] T 00000000000000000000000000000000 P 75c7c6919f434684b19eadeb15174ea9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:21:04.203966 27712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75c7c6919f434684b19eadeb15174ea9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:21:04.204229 27712 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75c7c6919f434684b19eadeb15174ea9 [term 1 LEADER]: Becoming Leader. State: Replica: 75c7c6919f434684b19eadeb15174ea9, State: Running, Role: LEADER
I20260811 17:21:04.204624 27712 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75c7c6919f434684b19eadeb15174ea9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c7c6919f434684b19eadeb15174ea9" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 36313 } }
I20260811 17:21:04.205238 27706 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75c7c6919f434684b19eadeb15174ea9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:21:04.206787 27713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75c7c6919f434684b19eadeb15174ea9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75c7c6919f434684b19eadeb15174ea9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c7c6919f434684b19eadeb15174ea9" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 36313 } } }
I20260811 17:21:04.206933 27713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75c7c6919f434684b19eadeb15174ea9 [sys.catalog]: This master's current role is: LEADER
I20260811 17:21:04.207443 27725 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:21:04.207406 27715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75c7c6919f434684b19eadeb15174ea9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75c7c6919f434684b19eadeb15174ea9. Latest consensus state: current_term: 1 leader_uuid: "75c7c6919f434684b19eadeb15174ea9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c7c6919f434684b19eadeb15174ea9" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 36313 } } }
I20260811 17:21:04.207681 27715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75c7c6919f434684b19eadeb15174ea9 [sys.catalog]: This master's current role is: LEADER
I20260811 17:21:04.208195 27706 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 17:21:04.211597 27725 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:21:04.223533 27725 catalog_manager.cc:1347] Generated new cluster ID: 288d8eaffd82430ba3168d55ec932e0b
I20260811 17:21:04.223686 27725 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:21:04.240820 27725 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:21:04.242012 27725 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:21:04.256409 27725 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75c7c6919f434684b19eadeb15174ea9: Generated new TSK 0
I20260811 17:21:04.257573 27725 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
I20260811 17:21:04.993070 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.157.148:56779/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:0
--local_ip_for_outbound_sockets=127.9.157.129
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:36313
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:46477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:21:05.141520 27764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:21:05.141891 27764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:21:05.141999 27764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:21:05.146585 27764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:21:05.146706 27764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:21:05.146811 27764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:21:05.152887 27764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:46477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.157.148:56779/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:36313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:21:05.154379 27764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:21:05.155608 27764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:21:05.167821 27771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:21:05.168570 27775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:21:05.168733 27773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:21:05.169831 27764 server_base.cc:1034] running on GCE node
I20260811 17:21:05.170430 27764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:21:05.171391 27764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:21:05.172712 27764 hybrid_clock.cc:648] HybridClock initialized: now 1786468865172665 us; error 55 us; skew 500 ppm
Aug 11 17:21:05 dist-test-slave-mjjr krb5kdc[26969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468865, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:21:05.176723 27764 init.cc:377] Logged in from keytab as kudu/127.9.157.129@KRBTEST.COM (short username kudu)
I20260811 17:21:05.178076 27764 webserver.cc:460] Webserver started at http://127.9.157.129:44171/ using document root <none> and password file <none>
I20260811 17:21:05.178711 27764 fs_manager.cc:359] Metadata directory not provided
I20260811 17:21:05.178809 27764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:21:05.179075 27764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 17:21:05 dist-test-slave-mjjr krb5kdc[26969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468865, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for HTTP/127.9.157.148@KRBTEST.COM
I20260811 17:21:05.199719 27764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-0/data/instance:
uuid: "73a18c73968143a8b96a140ae086d3cc"
format_stamp: "Formatted at 2026-08-11 17:21:05 on dist-test-slave-mjjr"
server_key: "04450a0978a5f9bbc8344538378d9f1d"
server_key_iv: "1de747fb382508989612606ea9036920"
server_key_version: "kuduclusterkey@0"
I20260811 17:21:05.200696 27764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance:
uuid: "73a18c73968143a8b96a140ae086d3cc"
format_stamp: "Formatted at 2026-08-11 17:21:05 on dist-test-slave-mjjr"
server_key: "04450a0978a5f9bbc8344538378d9f1d"
server_key_iv: "1de747fb382508989612606ea9036920"
server_key_version: "kuduclusterkey@0"
I20260811 17:21:05.207518 27764 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 17:21:05.230248 27787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:21:05.232160 27764 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260811 17:21:05.232363 27764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "73a18c73968143a8b96a140ae086d3cc"
format_stamp: "Formatted at 2026-08-11 17:21:05 on dist-test-slave-mjjr"
server_key: "04450a0978a5f9bbc8344538378d9f1d"
server_key_iv: "1de747fb382508989612606ea9036920"
server_key_version: "kuduclusterkey@0"
I20260811 17:21:05.232528 27764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:21:05.263860 27764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:21:05.268158 27764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:21:05.268471 27764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:21:05.269356 27764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:21:05.270573 27764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:21:05.270699 27764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:21:05.270840 27764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:21:05.270941 27764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:21:05.286573 27764 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:40307
I20260811 17:21:05.288089 27764 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:21:05.290356 27950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:40307 every 8 connection(s)
I20260811 17:21:05.297679 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 27764
I20260811 17:21:05.297842 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance
I20260811 17:21:05.298233 9846 external_mini_cluster.cc:1428] Setting key 2e6f2023528fd391e21e6f121da7b537
Aug 11 17:21:05 dist-test-slave-mjjr krb5kdc[26969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468865, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.157.130@KRBTEST.COM; defaulting to no policy
I20260811 17:21:05.322305 27953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:05.293250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:55127 (local address 127.9.157.190:36313)
0811 17:21:05.300423 (+ 7173us) server_negotiation.cc:206] Beginning negotiation
0811 17:21:05.300429 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:21:05.300449 (+ 20us) server_negotiation.cc:407] Connection header received
0811 17:21:05.304784 (+ 4335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:21:05.304793 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:21:05.304883 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:21:05.304969 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:21:05.307781 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:05.308738 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:05.310745 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:05.311160 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:05.311314 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:05.315321 (+ 4007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:21:05.315327 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:21:05.315330 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:21:05.315369 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:21:05.317633 (+ 2264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:21:05.318511 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:21:05.318516 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:21:05.318518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:21:05.318639 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:21:05.319163 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:21:05.319168 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:21:05.319170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:21:05.319469 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:21:05.319574 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 17:21:05.321635 (+ 2061us) server_negotiation.cc:299] Negotiation successful
0811 17:21:05.322052 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7081,"thread_start_us":734,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.130@KRBTEST.COM" created.
I20260811 17:21:05.324503 27954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:05.294000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:36313 (local address 127.9.157.129:55127)
0811 17:21:05.299495 (+ 5495us) negotiation.cc:107] Waiting for socket to connect
0811 17:21:05.299678 (+ 183us) client_negotiation.cc:174] Beginning negotiation
0811 17:21:05.304385 (+ 4707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:21:05.305744 (+ 1359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:21:05.305759 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:21:05.306373 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:21:05.307491 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:05.307523 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:05.308962 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:05.308966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:05.310536 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:05.310553 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:05.311570 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:05.311575 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:05.311945 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:05.312850 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:21:05.312878 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:21:05.315000 (+ 2122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:21:05.317846 (+ 2846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:21:05.317855 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:21:05.317879 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:21:05.318347 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:21:05.318789 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:21:05.318793 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:21:05.318795 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:21:05.319024 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:21:05.319959 (+ 935us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:21:05.319968 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:21:05.320434 (+ 466us) client_negotiation.cc:769] Sending connection context
0811 17:21:05.322733 (+ 2299us) client_negotiation.cc:240] Negotiation successful
0811 17:21:05.323252 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5180,"thread_start_us":1949,"threads_started":1}
I20260811 17:21:05.326649 27951 heartbeater.cc:347] Connected to a master server at 127.9.157.190:36313
I20260811 17:21:05.326938 27951 heartbeater.cc:464] Registering TS with master...
I20260811 17:21:05.327625 27951 heartbeater.cc:511] Master 127.9.157.190:36313 requested a full tablet report, sending...
I20260811 17:21:05.329973 27653 ts_manager.cc:194] Registered new tserver with Master: 73a18c73968143a8b96a140ae086d3cc (127.9.157.129:40307)
I20260811 17:21:05.331712 27653 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.129@KRBTEST.COM'} at 127.9.157.129:55127
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
I20260811 17:21:05.347205 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.157.148:56779/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:0
--local_ip_for_outbound_sockets=127.9.157.130
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:36313
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:46477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 17:21:05 dist-test-slave-mjjr krb5kdc[26969](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786468865, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 17:21:05.534893 27960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:21:05.535411 27960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:21:05.535502 27960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:21:05.542393 27960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:21:05.542515 27960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:21:05.542616 27960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:21:05.551698 27960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:46477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.157.148:56779/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:36313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:21:05.553432 27960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:21:05.555035 27960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:21:05.570147 27966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:21:05.570429 27970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:21:05.570688 27967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:21:05.571362 27960 server_base.cc:1034] running on GCE node
I20260811 17:21:05.572108 27960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:21:05.573168 27960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:21:05.576541 27960 hybrid_clock.cc:648] HybridClock initialized: now 1786468865576493 us; error 61 us; skew 500 ppm
Aug 11 17:21:05 dist-test-slave-mjjr krb5kdc[26969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468865, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:21:05.581262 27960 init.cc:377] Logged in from keytab as kudu/127.9.157.130@KRBTEST.COM (short username kudu)
I20260811 17:21:05.582751 27960 webserver.cc:460] Webserver started at http://127.9.157.130:36469/ using document root <none> and password file <none>
I20260811 17:21:05.583607 27960 fs_manager.cc:359] Metadata directory not provided
I20260811 17:21:05.583810 27960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:21:05.584128 27960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 17:21:05 dist-test-slave-mjjr krb5kdc[26969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468865, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for HTTP/127.9.157.148@KRBTEST.COM
I20260811 17:21:05.602757 27960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-1/data/instance:
uuid: "7bab2288998c4ec0a6f81790d9013867"
format_stamp: "Formatted at 2026-08-11 17:21:05 on dist-test-slave-mjjr"
server_key: "773a36a638c79309f3de7bb6f05f53ba"
server_key_iv: "7f0d795b9cb0c1e829741b2df8ef45df"
server_key_version: "kuduclusterkey@0"
I20260811 17:21:05.603809 27960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance:
uuid: "7bab2288998c4ec0a6f81790d9013867"
format_stamp: "Formatted at 2026-08-11 17:21:05 on dist-test-slave-mjjr"
server_key: "773a36a638c79309f3de7bb6f05f53ba"
server_key_iv: "7f0d795b9cb0c1e829741b2df8ef45df"
server_key_version: "kuduclusterkey@0"
I20260811 17:21:05.613487 27960 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.005s sys 0.000s
I20260811 17:21:05.647140 27977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:21:05.649183 27960 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.000s
I20260811 17:21:05.649467 27960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "7bab2288998c4ec0a6f81790d9013867"
format_stamp: "Formatted at 2026-08-11 17:21:05 on dist-test-slave-mjjr"
server_key: "773a36a638c79309f3de7bb6f05f53ba"
server_key_iv: "7f0d795b9cb0c1e829741b2df8ef45df"
server_key_version: "kuduclusterkey@0"
I20260811 17:21:05.649664 27960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:21:05.671177 27960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:21:05.676192 27960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:21:05.676487 27960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:21:05.677400 27960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:21:05.678715 27960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:21:05.678797 27960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:21:05.678949 27960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:21:05.678990 27960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:21:05.696060 27960 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:33365
I20260811 17:21:05.697639 27960 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:21:05.701011 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 27960
I20260811 17:21:05.701155 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance
I20260811 17:21:05.701511 9846 external_mini_cluster.cc:1428] Setting key 5d101c8c12edb923d9f4519cda757990
I20260811 17:21:05.699514 28133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:33365 every 8 connection(s)
Aug 11 17:21:05 dist-test-slave-mjjr krb5kdc[26969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468865, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.157.131@KRBTEST.COM; defaulting to no policy
I20260811 17:21:05.726293 27953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:05.704105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:55631 (local address 127.9.157.190:36313)
0811 17:21:05.704283 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 17:21:05.704289 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:21:05.704888 (+ 599us) server_negotiation.cc:407] Connection header received
0811 17:21:05.706506 (+ 1618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:21:05.706510 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:21:05.706595 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:21:05.706660 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:21:05.708811 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:05.709672 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:05.711839 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:05.712330 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:05.712452 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:05.720507 (+ 8055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:21:05.720514 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:21:05.720517 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:21:05.720559 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:21:05.722681 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:21:05.723678 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:21:05.723719 (+ 41us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:21:05.723721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:21:05.723844 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:21:05.724424 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:21:05.724430 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:21:05.724432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:21:05.724772 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:21:05.724910 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 17:21:05.725857 (+ 947us) server_negotiation.cc:299] Negotiation successful
0811 17:21:05.726083 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.131@KRBTEST.COM" created.
I20260811 17:21:05.728368 28136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:05.702717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:36313 (local address 127.9.157.130:55631)
0811 17:21:05.704715 (+ 1998us) negotiation.cc:107] Waiting for socket to connect
0811 17:21:05.704783 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 17:21:05.706186 (+ 1403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:21:05.707204 (+ 1018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:21:05.707218 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:21:05.707686 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:21:05.708576 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:05.708602 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:05.709887 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:05.709891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:05.711612 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:05.711629 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:05.712733 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:05.712737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:05.713004 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:05.718024 (+ 5020us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:21:05.718057 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:21:05.720161 (+ 2104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:21:05.722947 (+ 2786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:21:05.722957 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:21:05.722984 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:21:05.723478 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:21:05.724008 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:21:05.724012 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:21:05.724014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:21:05.724262 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:21:05.725036 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:21:05.725045 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:21:05.725471 (+ 426us) client_negotiation.cc:769] Sending connection context
0811 17:21:05.726646 (+ 1175us) client_negotiation.cc:240] Negotiation successful
0811 17:21:05.727111 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1727,"thread_start_us":208,"threads_started":1}
I20260811 17:21:05.730052 28134 heartbeater.cc:347] Connected to a master server at 127.9.157.190:36313
I20260811 17:21:05.730348 28134 heartbeater.cc:464] Registering TS with master...
I20260811 17:21:05.731081 28134 heartbeater.cc:511] Master 127.9.157.190:36313 requested a full tablet report, sending...
I20260811 17:21:05.732548 27653 ts_manager.cc:194] Registered new tserver with Master: 7bab2288998c4ec0a6f81790d9013867 (127.9.157.130:33365)
I20260811 17:21:05.733422 27653 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.130@KRBTEST.COM'} at 127.9.157.130:55631
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
I20260811 17:21:05.759320 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.157.148:56779/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:0
--local_ip_for_outbound_sockets=127.9.157.131
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:36313
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:46477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:21:05.975752 28141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:21:05.976182 28141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:21:05.976308 28141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:21:05.982519 28141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:21:05.982702 28141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:21:05.982853 28141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:21:05.991253 28141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:46477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.157.148:56779/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:36313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:21:05.993112 28141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:21:05.994668 28141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:21:06.009642 28141 server_base.cc:1034] running on GCE node
W20260811 17:21:06.010298 28146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:21:06.010720 28149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:21:06.011011 28147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:21:06.011610 28141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:21:06.012534 28141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:21:06.015836 28141 hybrid_clock.cc:648] HybridClock initialized: now 1786468866015747 us; error 93 us; skew 500 ppm
Aug 11 17:21:06 dist-test-slave-mjjr krb5kdc[26969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468866, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:21:06.020210 28141 init.cc:377] Logged in from keytab as kudu/127.9.157.131@KRBTEST.COM (short username kudu)
I20260811 17:21:06.021565 28141 webserver.cc:460] Webserver started at http://127.9.157.131:45403/ using document root <none> and password file <none>
I20260811 17:21:06.022403 28141 fs_manager.cc:359] Metadata directory not provided
I20260811 17:21:06.022562 28141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:21:06.022882 28141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 17:21:06 dist-test-slave-mjjr krb5kdc[26969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468866, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for HTTP/127.9.157.148@KRBTEST.COM
I20260811 17:21:06.041514 28141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-2/data/instance:
uuid: "6a9260db598d4f79a0bbc5e172f91d6e"
format_stamp: "Formatted at 2026-08-11 17:21:06 on dist-test-slave-mjjr"
server_key: "d0572831fc5b6461926051976a92170a"
server_key_iv: "9ae174d04bc384183af42cee2d05983e"
server_key_version: "kuduclusterkey@0"
I20260811 17:21:06.042766 28141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance:
uuid: "6a9260db598d4f79a0bbc5e172f91d6e"
format_stamp: "Formatted at 2026-08-11 17:21:06 on dist-test-slave-mjjr"
server_key: "d0572831fc5b6461926051976a92170a"
server_key_iv: "9ae174d04bc384183af42cee2d05983e"
server_key_version: "kuduclusterkey@0"
I20260811 17:21:06.054754 28141 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.000s sys 0.004s
Aug 11 17:21:06 dist-test-slave-mjjr krb5kdc[26969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468865, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for HTTP/127.9.157.148@KRBTEST.COM
I20260811 17:21:06.083765 28159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:21:06.085681 28141 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.002s
I20260811 17:21:06.085846 28141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "6a9260db598d4f79a0bbc5e172f91d6e"
format_stamp: "Formatted at 2026-08-11 17:21:06 on dist-test-slave-mjjr"
server_key: "d0572831fc5b6461926051976a92170a"
server_key_iv: "9ae174d04bc384183af42cee2d05983e"
server_key_version: "kuduclusterkey@0"
I20260811 17:21:06.086000 28141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:21:06.120338 28141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:21:06.124748 28141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:21:06.125067 28141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:21:06.125932 28141 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:21:06.127246 28141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:21:06.127370 28141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 17:21:06.127482 28141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:21:06.127554 28141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:21:06.140269 28141 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:40771
I20260811 17:21:06.141697 28141 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:21:06.146386 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 28141
I20260811 17:21:06.146582 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance
I20260811 17:21:06.146994 9846 external_mini_cluster.cc:1428] Setting key fa7d021bd6714e4bb84a7bbd40b83d20
I20260811 17:21:06.152943 28325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:40771 every 8 connection(s)
Aug 11 17:21:06 dist-test-slave-mjjr krb5kdc[26969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468866, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:21:06.175518 27953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:06.152560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:43039 (local address 127.9.157.190:36313)
0811 17:21:06.152710 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 17:21:06.152716 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:21:06.155208 (+ 2492us) server_negotiation.cc:407] Connection header received
0811 17:21:06.156682 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:21:06.156686 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:21:06.156759 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:21:06.156824 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:21:06.158823 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:06.159746 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:06.163894 (+ 4148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:06.164310 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:06.164413 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:06.168178 (+ 3765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:21:06.168184 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:21:06.168200 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:21:06.168236 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:21:06.170310 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:21:06.171136 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:21:06.171141 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:21:06.171143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:21:06.171259 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:21:06.171975 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:21:06.171979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:21:06.171981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:21:06.172263 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:21:06.172362 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 17:21:06.175139 (+ 2777us) server_negotiation.cc:299] Negotiation successful
0811 17:21:06.175329 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 17:21:06.177747 28328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:06.151197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:36313 (local address 127.9.157.131:43039)
0811 17:21:06.154966 (+ 3769us) negotiation.cc:107] Waiting for socket to connect
0811 17:21:06.155049 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0811 17:21:06.156360 (+ 1311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:21:06.157349 (+ 989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:21:06.157361 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:21:06.157791 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:21:06.158597 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:06.158620 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:06.162050 (+ 3430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:06.162054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:06.163704 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:06.163723 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:06.164701 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:06.164706 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:06.164958 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:06.165855 (+ 897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:21:06.165882 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:21:06.167898 (+ 2016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:21:06.170527 (+ 2629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:21:06.170535 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:21:06.170559 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:21:06.170989 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:21:06.171391 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:21:06.171394 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:21:06.171396 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:21:06.171814 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:21:06.174347 (+ 2533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:21:06.174357 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:21:06.174795 (+ 438us) client_negotiation.cc:769] Sending connection context
0811 17:21:06.176086 (+ 1291us) client_negotiation.cc:240] Negotiation successful
0811 17:21:06.176537 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3457,"thread_start_us":218,"threads_started":1}
I20260811 17:21:06.179634 28326 heartbeater.cc:347] Connected to a master server at 127.9.157.190:36313
I20260811 17:21:06.179955 28326 heartbeater.cc:464] Registering TS with master...
I20260811 17:21:06.180657 28326 heartbeater.cc:511] Master 127.9.157.190:36313 requested a full tablet report, sending...
I20260811 17:21:06.182106 27653 ts_manager.cc:194] Registered new tserver with Master: 6a9260db598d4f79a0bbc5e172f91d6e (127.9.157.131:40771)
I20260811 17:21:06.182920 27653 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.131@KRBTEST.COM'} at 127.9.157.131:43039
I20260811 17:21:06.187630 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:21:06.202689 27953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:06.190773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53804 (local address 127.9.157.190:36313)
0811 17:21:06.190909 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 17:21:06.190915 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:21:06.191744 (+ 829us) server_negotiation.cc:407] Connection header received
0811 17:21:06.192032 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:21:06.192036 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:21:06.192112 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:21:06.192173 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:21:06.193654 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:06.194507 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:06.195841 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:06.196205 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:06.196293 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:06.198419 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:21:06.198424 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:21:06.198427 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:21:06.198462 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:21:06.200421 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:21:06.201014 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:21:06.201018 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:21:06.201020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:21:06.201119 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:21:06.201483 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:21:06.201487 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:21:06.201489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:21:06.201765 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:21:06.201884 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 17:21:06.202344 (+ 460us) server_negotiation.cc:299] Negotiation successful
0811 17:21:06.202487 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 17:21:06.207487 27653 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53804:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:21:06.212463 27653 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:21:06.246913 28343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:06.232369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:40771 (local address 127.0.0.1:54014)
0811 17:21:06.236115 (+ 3746us) negotiation.cc:107] Waiting for socket to connect
0811 17:21:06.236132 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:21:06.236258 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:21:06.238371 (+ 2113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:21:06.238378 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:21:06.238409 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:21:06.238605 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:06.238615 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:06.241853 (+ 3238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:06.241857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:06.243331 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:06.243347 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:06.244906 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:06.244911 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:06.245069 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:06.245117 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 17:21:06.245266 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 17:21:06.245451 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3672,"thread_start_us":112,"threads_started":1}
I20260811 17:21:06.248080 28342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:06.231959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54014 (local address 127.9.157.131:40771)
0811 17:21:06.235941 (+ 3982us) server_negotiation.cc:206] Beginning negotiation
0811 17:21:06.235956 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 17:21:06.236414 (+ 458us) server_negotiation.cc:407] Connection header received
0811 17:21:06.237603 (+ 1189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:21:06.237613 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:21:06.237911 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:21:06.238120 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:21:06.239151 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:06.240547 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:06.243554 (+ 3007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:06.244502 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:06.244633 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:06.244826 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 17:21:06.247747 (+ 2921us) server_negotiation.cc:299] Negotiation successful
0811 17:21:06.247902 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3837,"thread_start_us":149,"threads_started":1}
I20260811 17:21:06.252105 28225 tablet_service.cc:1467] Processing CreateTablet for tablet eaef9ec111754bbf83a7f8c4773f1b29 (DEFAULT_TABLE table=test-table [id=e21aad275b2148b981e218c1c1fd0c12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:21:06.253597 28225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eaef9ec111754bbf83a7f8c4773f1b29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:21:06.262984 28340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:06.234405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:33365 (local address 127.0.0.1:57050)
0811 17:21:06.238894 (+ 4489us) negotiation.cc:107] Waiting for socket to connect
0811 17:21:06.238911 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:21:06.239005 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:21:06.250963 (+ 11958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:21:06.250967 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:21:06.251001 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:21:06.251162 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:06.251172 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:06.255771 (+ 4599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:06.255775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:06.256970 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:06.256986 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:06.262443 (+ 5457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:06.262448 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:06.262574 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:06.262616 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 17:21:06.262686 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 17:21:06.262786 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4415,"thread_start_us":152,"threads_started":1}
I20260811 17:21:06.264385 28346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:06.234616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57050 (local address 127.9.157.130:33365)
0811 17:21:06.249665 (+ 15049us) server_negotiation.cc:206] Beginning negotiation
0811 17:21:06.249674 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 17:21:06.249708 (+ 34us) server_negotiation.cc:407] Connection header received
0811 17:21:06.249808 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:21:06.249814 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:21:06.250054 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:21:06.251385 (+ 1331us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:21:06.252801 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:06.253958 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:06.257179 (+ 3221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:06.258070 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:06.263709 (+ 5639us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:06.263944 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 17:21:06.264024 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 17:21:06.264161 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14907,"thread_start_us":66,"threads_started":1}
I20260811 17:21:06.265293 28344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:06.229003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:40307 (local address 127.0.0.1:50172)
0811 17:21:06.235286 (+ 6283us) negotiation.cc:107] Waiting for socket to connect
0811 17:21:06.235368 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0811 17:21:06.235541 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:21:06.254160 (+ 18619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:21:06.254164 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:21:06.254209 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:21:06.254333 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:06.254341 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:06.258568 (+ 4227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:06.258572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:06.259809 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:06.259825 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:06.264775 (+ 4950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:06.264780 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:06.264902 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:06.264939 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 17:21:06.265015 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 17:21:06.265110 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6011,"thread_start_us":169,"threads_started":1}
I20260811 17:21:06.266499 28341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:06.229108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50172 (local address 127.9.157.129:40307)
0811 17:21:06.250434 (+ 21326us) server_negotiation.cc:206] Beginning negotiation
0811 17:21:06.250442 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:21:06.250469 (+ 27us) server_negotiation.cc:407] Connection header received
0811 17:21:06.250552 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:21:06.250557 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:21:06.250749 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:21:06.254538 (+ 3789us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:21:06.254728 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:06.258401 (+ 3673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:06.261269 (+ 2868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:06.262232 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:06.265865 (+ 3633us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:06.266091 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 17:21:06.266161 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 17:21:06.266299 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21189,"thread_start_us":124,"threads_started":1}
I20260811 17:21:06.267585 28347 tablet_bootstrap.cc:492] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e: Bootstrap starting.
I20260811 17:21:06.270458 27851 tablet_service.cc:1467] Processing CreateTablet for tablet eaef9ec111754bbf83a7f8c4773f1b29 (DEFAULT_TABLE table=test-table [id=e21aad275b2148b981e218c1c1fd0c12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:21:06.270582 28043 tablet_service.cc:1467] Processing CreateTablet for tablet eaef9ec111754bbf83a7f8c4773f1b29 (DEFAULT_TABLE table=test-table [id=e21aad275b2148b981e218c1c1fd0c12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:21:06.271063 28347 tablet_bootstrap.cc:654] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e: Neither blocks nor log segments found. Creating new log.
I20260811 17:21:06.271862 28043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eaef9ec111754bbf83a7f8c4773f1b29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:21:06.271881 27851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eaef9ec111754bbf83a7f8c4773f1b29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:21:06.272190 28347 log.cc:824] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e: Log is configured to *not* fsync() on all Append() calls
I20260811 17:21:06.274737 28347 tablet_bootstrap.cc:492] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e: No bootstrap required, opened a new log
I20260811 17:21:06.275008 28347 ts_tablet_manager.cc:1397] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 17:21:06.279409 28347 raft_consensus.cc:348] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73a18c73968143a8b96a140ae086d3cc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40307 } } peers { permanent_uuid: "6a9260db598d4f79a0bbc5e172f91d6e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40771 } } peers { permanent_uuid: "7bab2288998c4ec0a6f81790d9013867" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 33365 } }
I20260811 17:21:06.279870 28347 raft_consensus.cc:374] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:21:06.279966 28347 raft_consensus.cc:729] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a9260db598d4f79a0bbc5e172f91d6e, State: Initialized, Role: FOLLOWER
I20260811 17:21:06.280526 28347 consensus_queue.cc:260] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73a18c73968143a8b96a140ae086d3cc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40307 } } peers { permanent_uuid: "6a9260db598d4f79a0bbc5e172f91d6e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40771 } } peers { permanent_uuid: "7bab2288998c4ec0a6f81790d9013867" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 33365 } }
I20260811 17:21:06.281735 28347 ts_tablet_manager.cc:1428] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260811 17:21:06.282243 28326 heartbeater.cc:503] Master 127.9.157.190:36313 was elected leader, sending a full tablet report...
I20260811 17:21:06.282452 28350 tablet_bootstrap.cc:492] T eaef9ec111754bbf83a7f8c4773f1b29 P 73a18c73968143a8b96a140ae086d3cc: Bootstrap starting.
I20260811 17:21:06.285704 28350 tablet_bootstrap.cc:654] T eaef9ec111754bbf83a7f8c4773f1b29 P 73a18c73968143a8b96a140ae086d3cc: Neither blocks nor log segments found. Creating new log.
I20260811 17:21:06.287103 28350 log.cc:824] T eaef9ec111754bbf83a7f8c4773f1b29 P 73a18c73968143a8b96a140ae086d3cc: Log is configured to *not* fsync() on all Append() calls
I20260811 17:21:06.288203 28353 tablet_bootstrap.cc:492] T eaef9ec111754bbf83a7f8c4773f1b29 P 7bab2288998c4ec0a6f81790d9013867: Bootstrap starting.
I20260811 17:21:06.291592 28353 tablet_bootstrap.cc:654] T eaef9ec111754bbf83a7f8c4773f1b29 P 7bab2288998c4ec0a6f81790d9013867: Neither blocks nor log segments found. Creating new log.
I20260811 17:21:06.291950 28350 tablet_bootstrap.cc:492] T eaef9ec111754bbf83a7f8c4773f1b29 P 73a18c73968143a8b96a140ae086d3cc: No bootstrap required, opened a new log
I20260811 17:21:06.292307 28350 ts_tablet_manager.cc:1397] T eaef9ec111754bbf83a7f8c4773f1b29 P 73a18c73968143a8b96a140ae086d3cc: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260811 17:21:06.293073 28353 log.cc:824] T eaef9ec111754bbf83a7f8c4773f1b29 P 7bab2288998c4ec0a6f81790d9013867: Log is configured to *not* fsync() on all Append() calls
I20260811 17:21:06.296108 28353 tablet_bootstrap.cc:492] T eaef9ec111754bbf83a7f8c4773f1b29 P 7bab2288998c4ec0a6f81790d9013867: No bootstrap required, opened a new log
I20260811 17:21:06.296384 28353 ts_tablet_manager.cc:1397] T eaef9ec111754bbf83a7f8c4773f1b29 P 7bab2288998c4ec0a6f81790d9013867: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 17:21:06.297857 28350 raft_consensus.cc:348] T eaef9ec111754bbf83a7f8c4773f1b29 P 73a18c73968143a8b96a140ae086d3cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73a18c73968143a8b96a140ae086d3cc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40307 } } peers { permanent_uuid: "6a9260db598d4f79a0bbc5e172f91d6e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40771 } } peers { permanent_uuid: "7bab2288998c4ec0a6f81790d9013867" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 33365 } }
I20260811 17:21:06.298202 28350 raft_consensus.cc:374] T eaef9ec111754bbf83a7f8c4773f1b29 P 73a18c73968143a8b96a140ae086d3cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:21:06.298255 28350 raft_consensus.cc:729] T eaef9ec111754bbf83a7f8c4773f1b29 P 73a18c73968143a8b96a140ae086d3cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73a18c73968143a8b96a140ae086d3cc, State: Initialized, Role: FOLLOWER
I20260811 17:21:06.298874 28350 consensus_queue.cc:260] T eaef9ec111754bbf83a7f8c4773f1b29 P 73a18c73968143a8b96a140ae086d3cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73a18c73968143a8b96a140ae086d3cc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40307 } } peers { permanent_uuid: "6a9260db598d4f79a0bbc5e172f91d6e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40771 } } peers { permanent_uuid: "7bab2288998c4ec0a6f81790d9013867" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 33365 } }
I20260811 17:21:06.300217 28350 ts_tablet_manager.cc:1428] T eaef9ec111754bbf83a7f8c4773f1b29 P 73a18c73968143a8b96a140ae086d3cc: Time spent starting tablet: real 0.008s user 0.005s sys 0.002s
I20260811 17:21:06.301225 28351 raft_consensus.cc:482] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:21:06.301446 28351 raft_consensus.cc:504] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73a18c73968143a8b96a140ae086d3cc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40307 } } peers { permanent_uuid: "6a9260db598d4f79a0bbc5e172f91d6e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40771 } } peers { permanent_uuid: "7bab2288998c4ec0a6f81790d9013867" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 33365 } }
I20260811 17:21:06.301491 28353 raft_consensus.cc:348] T eaef9ec111754bbf83a7f8c4773f1b29 P 7bab2288998c4ec0a6f81790d9013867 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73a18c73968143a8b96a140ae086d3cc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40307 } } peers { permanent_uuid: "6a9260db598d4f79a0bbc5e172f91d6e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40771 } } peers { permanent_uuid: "7bab2288998c4ec0a6f81790d9013867" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 33365 } }
I20260811 17:21:06.301810 28353 raft_consensus.cc:374] T eaef9ec111754bbf83a7f8c4773f1b29 P 7bab2288998c4ec0a6f81790d9013867 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:21:06.301898 28353 raft_consensus.cc:729] T eaef9ec111754bbf83a7f8c4773f1b29 P 7bab2288998c4ec0a6f81790d9013867 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bab2288998c4ec0a6f81790d9013867, State: Initialized, Role: FOLLOWER
I20260811 17:21:06.302106 27951 heartbeater.cc:503] Master 127.9.157.190:36313 was elected leader, sending a full tablet report...
I20260811 17:21:06.302779 28351 leader_election.cc:288] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73a18c73968143a8b96a140ae086d3cc (127.9.157.129:40307), 7bab2288998c4ec0a6f81790d9013867 (127.9.157.130:33365)
I20260811 17:21:06.303046 28353 consensus_queue.cc:260] T eaef9ec111754bbf83a7f8c4773f1b29 P 7bab2288998c4ec0a6f81790d9013867 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73a18c73968143a8b96a140ae086d3cc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40307 } } peers { permanent_uuid: "6a9260db598d4f79a0bbc5e172f91d6e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40771 } } peers { permanent_uuid: "7bab2288998c4ec0a6f81790d9013867" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 33365 } }
I20260811 17:21:06.304742 28353 ts_tablet_manager.cc:1428] T eaef9ec111754bbf83a7f8c4773f1b29 P 7bab2288998c4ec0a6f81790d9013867: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260811 17:21:06.305323 28134 heartbeater.cc:503] Master 127.9.157.190:36313 was elected leader, sending a full tablet report...
I20260811 17:21:06.316514 28328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:06.305978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:33365 (local address 127.9.157.131:45125)
0811 17:21:06.306813 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0811 17:21:06.306832 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:21:06.306951 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:21:06.312421 (+ 5470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:21:06.312426 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:21:06.312453 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:21:06.312582 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:06.312589 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:06.313634 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:06.313638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:06.314914 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:06.314930 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:06.315964 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:06.315969 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:06.316105 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:06.316141 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 17:21:06.316240 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 17:21:06.316343 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743}
I20260811 17:21:06.318740 28346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:06.311902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:45125 (local address 127.9.157.130:33365)
0811 17:21:06.312031 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 17:21:06.312036 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:21:06.312054 (+ 18us) server_negotiation.cc:407] Connection header received
0811 17:21:06.312114 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:21:06.312117 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:21:06.312168 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:21:06.312262 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:21:06.312723 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:06.313479 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:06.315079 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:06.315703 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:06.315818 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:06.315873 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 17:21:06.318448 (+ 2575us) server_negotiation.cc:299] Negotiation successful
0811 17:21:06.318564 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 17:21:06.319731 28072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaef9ec111754bbf83a7f8c4773f1b29" candidate_uuid: "6a9260db598d4f79a0bbc5e172f91d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bab2288998c4ec0a6f81790d9013867" is_pre_election: true
I20260811 17:21:06.320124 28072 raft_consensus.cc:2454] T eaef9ec111754bbf83a7f8c4773f1b29 P 7bab2288998c4ec0a6f81790d9013867 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a9260db598d4f79a0bbc5e172f91d6e in term 0.
I20260811 17:21:06.320779 28160 leader_election.cc:302] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a9260db598d4f79a0bbc5e172f91d6e, 7bab2288998c4ec0a6f81790d9013867; no voters:
I20260811 17:21:06.321152 28351 raft_consensus.cc:2781] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:21:06.321249 28351 raft_consensus.cc:482] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:21:06.321295 28351 raft_consensus.cc:3037] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:21:06.322758 28351 raft_consensus.cc:504] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73a18c73968143a8b96a140ae086d3cc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40307 } } peers { permanent_uuid: "6a9260db598d4f79a0bbc5e172f91d6e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40771 } } peers { permanent_uuid: "7bab2288998c4ec0a6f81790d9013867" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 33365 } }
I20260811 17:21:06.323192 28351 leader_election.cc:288] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e [CANDIDATE]: Term 1 election: Requested vote from peers 73a18c73968143a8b96a140ae086d3cc (127.9.157.129:40307), 7bab2288998c4ec0a6f81790d9013867 (127.9.157.130:33365)
I20260811 17:21:06.323884 28072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaef9ec111754bbf83a7f8c4773f1b29" candidate_uuid: "6a9260db598d4f79a0bbc5e172f91d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bab2288998c4ec0a6f81790d9013867"
I20260811 17:21:06.324039 28072 raft_consensus.cc:3037] T eaef9ec111754bbf83a7f8c4773f1b29 P 7bab2288998c4ec0a6f81790d9013867 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:21:06.325533 28072 raft_consensus.cc:2454] T eaef9ec111754bbf83a7f8c4773f1b29 P 7bab2288998c4ec0a6f81790d9013867 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a9260db598d4f79a0bbc5e172f91d6e in term 1.
I20260811 17:21:06.326036 28160 leader_election.cc:302] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a9260db598d4f79a0bbc5e172f91d6e, 7bab2288998c4ec0a6f81790d9013867; no voters:
I20260811 17:21:06.326256 28351 raft_consensus.cc:2781] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:21:06.326360 28341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:06.310490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:38891 (local address 127.9.157.129:40307)
0811 17:21:06.310679 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 17:21:06.310695 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 17:21:06.317257 (+ 6562us) server_negotiation.cc:407] Connection header received
0811 17:21:06.317332 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:21:06.317335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:21:06.317395 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:21:06.317481 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:21:06.319721 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:06.320536 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:06.324415 (+ 3879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:06.325107 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:06.325256 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:06.325342 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 17:21:06.326104 (+ 762us) server_negotiation.cc:299] Negotiation successful
0811 17:21:06.326209 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 17:21:06.326537 28351 raft_consensus.cc:686] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e [term 1 LEADER]: Becoming Leader. State: Replica: 6a9260db598d4f79a0bbc5e172f91d6e, State: Running, Role: LEADER
I20260811 17:21:06.326982 28351 consensus_queue.cc:237] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73a18c73968143a8b96a140ae086d3cc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40307 } } peers { permanent_uuid: "6a9260db598d4f79a0bbc5e172f91d6e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40771 } } peers { permanent_uuid: "7bab2288998c4ec0a6f81790d9013867" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 33365 } }
I20260811 17:21:06.331140 27653 catalog_manager.cc:5495] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a9260db598d4f79a0bbc5e172f91d6e (127.9.157.131). New cstate: current_term: 1 leader_uuid: "6a9260db598d4f79a0bbc5e172f91d6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73a18c73968143a8b96a140ae086d3cc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a9260db598d4f79a0bbc5e172f91d6e" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bab2288998c4ec0a6f81790d9013867" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 33365 } health_report { overall_health: UNKNOWN } } }
I20260811 17:21:06.335322 28328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:06.306377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:40307 (local address 127.9.157.131:38891)
0811 17:21:06.316762 (+ 10385us) negotiation.cc:107] Waiting for socket to connect
0811 17:21:06.316783 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 17:21:06.316889 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:21:06.319342 (+ 2453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:21:06.319347 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:21:06.319375 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:21:06.319492 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:06.319501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:06.320825 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:06.320829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:06.324220 (+ 3391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:06.324236 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:06.325518 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:06.325523 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:06.325719 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:06.325755 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 17:21:06.325848 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 17:21:06.325945 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10291,"thread_start_us":175,"threads_started":1}
I20260811 17:21:06.336362 27866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaef9ec111754bbf83a7f8c4773f1b29" candidate_uuid: "6a9260db598d4f79a0bbc5e172f91d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73a18c73968143a8b96a140ae086d3cc" is_pre_election: true
I20260811 17:21:06.336721 27866 raft_consensus.cc:2454] T eaef9ec111754bbf83a7f8c4773f1b29 P 73a18c73968143a8b96a140ae086d3cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a9260db598d4f79a0bbc5e172f91d6e in term 0.
I20260811 17:21:06.337088 27885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaef9ec111754bbf83a7f8c4773f1b29" candidate_uuid: "6a9260db598d4f79a0bbc5e172f91d6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73a18c73968143a8b96a140ae086d3cc"
I20260811 17:21:06.337195 27885 raft_consensus.cc:3037] T eaef9ec111754bbf83a7f8c4773f1b29 P 73a18c73968143a8b96a140ae086d3cc [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:21:06.338647 27885 raft_consensus.cc:2454] T eaef9ec111754bbf83a7f8c4773f1b29 P 73a18c73968143a8b96a140ae086d3cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a9260db598d4f79a0bbc5e172f91d6e in term 1.
I20260811 17:21:06.352615 28342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:06.347236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54024 (local address 127.9.157.131:40771)
0811 17:21:06.347395 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 17:21:06.347400 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:21:06.347419 (+ 19us) server_negotiation.cc:407] Connection header received
0811 17:21:06.347615 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:21:06.347619 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:21:06.347727 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:21:06.347850 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:21:06.348211 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:06.348961 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:06.350290 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:06.350627 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:06.350745 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:06.351195 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:21:06.351896 (+ 701us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:21:06.352029 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 17:21:06.352169 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 17:21:06.352250 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 17:21:06.361184 28072 raft_consensus.cc:1261] T eaef9ec111754bbf83a7f8c4773f1b29 P 7bab2288998c4ec0a6f81790d9013867 [term 1 FOLLOWER]: Refusing update from remote peer 6a9260db598d4f79a0bbc5e172f91d6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:21:06.362363 27885 raft_consensus.cc:1261] T eaef9ec111754bbf83a7f8c4773f1b29 P 73a18c73968143a8b96a140ae086d3cc [term 1 FOLLOWER]: Refusing update from remote peer 6a9260db598d4f79a0bbc5e172f91d6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:21:06.362821 28363 consensus_queue.cc:1035] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bab2288998c4ec0a6f81790d9013867" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 33365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:21:06.363341 28363 consensus_queue.cc:1035] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "73a18c73968143a8b96a140ae086d3cc" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:21:06.377375 28366 mvcc.cc:205] Tried to move back new op lower bound from 7317376476605456384 to 7317376476479647744. Current Snapshot: MvccSnapshot[applied={T|T < 7317376476605456384}]
I20260811 17:21:06.380151 28371 mvcc.cc:205] Tried to move back new op lower bound from 7317376476605456384 to 7317376476479647744. Current Snapshot: MvccSnapshot[applied={T|T < 7317376476605456384}]
I20260811 17:21:06.391376 28370 mvcc.cc:205] Tried to move back new op lower bound from 7317376476605456384 to 7317376476479647744. Current Snapshot: MvccSnapshot[applied={T|T < 7317376476605456384}]
I20260811 17:21:06.396180 27652 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:21:06.396411 27652 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:21:06.399613 27652 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 75c7c6919f434684b19eadeb15174ea9: Sending DeleteTablet for 3 replicas of tablet eaef9ec111754bbf83a7f8c4773f1b29
I20260811 17:21:06.401342 28225 tablet_service.cc:1514] Processing DeleteTablet for tablet eaef9ec111754bbf83a7f8c4773f1b29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:21:06 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:54014
I20260811 17:21:06.401603 27851 tablet_service.cc:1514] Processing DeleteTablet for tablet eaef9ec111754bbf83a7f8c4773f1b29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:21:06 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:50172
I20260811 17:21:06.402143 28043 tablet_service.cc:1514] Processing DeleteTablet for tablet eaef9ec111754bbf83a7f8c4773f1b29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:21:06 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:57050
I20260811 17:21:06.405304 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 27764
I20260811 17:21:06.418546 28384 tablet_replica.cc:330] T eaef9ec111754bbf83a7f8c4773f1b29 P 7bab2288998c4ec0a6f81790d9013867: stopping tablet replica
I20260811 17:21:06.419204 28384 raft_consensus.cc:2229] T eaef9ec111754bbf83a7f8c4773f1b29 P 7bab2288998c4ec0a6f81790d9013867 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:21:06.419798 28384 raft_consensus.cc:2258] T eaef9ec111754bbf83a7f8c4773f1b29 P 7bab2288998c4ec0a6f81790d9013867 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:21:06.420631 28380 tablet_replica.cc:330] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e: stopping tablet replica
I20260811 17:21:06.421186 28380 raft_consensus.cc:2229] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:21:06.421767 28380 raft_consensus.cc:2258] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:21:06.424988 28380 ts_tablet_manager.cc:1905] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:21:06.429579 28380 ts_tablet_manager.cc:1918] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:21:06.429790 28380 log.cc:1196] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-2/wal/wals/eaef9ec111754bbf83a7f8c4773f1b29
I20260811 17:21:06.430239 28380 ts_tablet_manager.cc:1939] T eaef9ec111754bbf83a7f8c4773f1b29 P 6a9260db598d4f79a0bbc5e172f91d6e: Deleting consensus metadata
I20260811 17:21:06.433064 28384 ts_tablet_manager.cc:1905] T eaef9ec111754bbf83a7f8c4773f1b29 P 7bab2288998c4ec0a6f81790d9013867: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 17:21:06.433666 27630 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.157.129:40307 (error 108)
I20260811 17:21:06.435478 28344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:06.435048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:40307 (local address 127.0.0.1:50180)
0811 17:21:06.435269 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 17:21:06.435347 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.129:40307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260811 17:21:06.436456 28384 ts_tablet_manager.cc:1918] T eaef9ec111754bbf83a7f8c4773f1b29 P 7bab2288998c4ec0a6f81790d9013867: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:21:06.436587 28384 log.cc:1196] T eaef9ec111754bbf83a7f8c4773f1b29 P 7bab2288998c4ec0a6f81790d9013867: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786468743878666-9846-0/minicluster-data/ts-1/wal/wals/eaef9ec111754bbf83a7f8c4773f1b29
I20260811 17:21:06.436972 28384 ts_tablet_manager.cc:1939] T eaef9ec111754bbf83a7f8c4773f1b29 P 7bab2288998c4ec0a6f81790d9013867: Deleting consensus metadata
W20260811 17:21:06.438208 27630 catalog_manager.cc:4584] TS 73a18c73968143a8b96a140ae086d3cc (127.9.157.129:40307): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eaef9ec111754bbf83a7f8c4773f1b29: Network error: Client connection negotiation failed: client connection to 127.9.157.129:40307: connect: Connection refused (error 111)
I20260811 17:21:06.438592 27628 catalog_manager.cc:4841] TS 7bab2288998c4ec0a6f81790d9013867 (127.9.157.130:33365): tablet eaef9ec111754bbf83a7f8c4773f1b29 (table test-table [id=e21aad275b2148b981e218c1c1fd0c12]) successfully deleted
I20260811 17:21:06.438639 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 27960
I20260811 17:21:06.453344 27630 catalog_manager.cc:4841] TS 6a9260db598d4f79a0bbc5e172f91d6e (127.9.157.131:40771): tablet eaef9ec111754bbf83a7f8c4773f1b29 (table test-table [id=e21aad275b2148b981e218c1c1fd0c12]) successfully deleted
I20260811 17:21:06.462648 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 28141
I20260811 17:21:06.491945 28344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:06.491452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:40307 (local address 127.0.0.1:50192)
0811 17:21:06.491738 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 17:21:06.491823 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.129:40307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
I20260811 17:21:06.493140 28344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:06.492804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:40307 (local address 127.0.0.1:50196)
0811 17:21:06.492977 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 17:21:06.493044 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.129:40307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260811 17:21:06.496587 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 27600
I20260811 17:21:06.520540 9846 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 17:21:06.866940 9846 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 17:21:06.867081 9846 mini_ranger.cc:66] Stopping Ranger...
I20260811 17:21:07.223949 9846 mini_ranger.cc:68] Stopped Ranger
2026-08-11 17:21:07.224 UTC [27010] LOG: received smart shutdown request
2026-08-11 17:21:07.228 UTC [27010] LOG: background worker "logical replication launcher" (PID 27021) exited with exit code 1
2026-08-11 17:21:07.228 UTC [27015] LOG: shutting down
2026-08-11 17:21:07.432 UTC [27010] LOG: database system is shut down
2026-08-11T17:21:07Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (52189 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:21:07 dist-test-slave-mjjr krb5kdc[28396](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:21:07 dist-test-slave-mjjr krb5kdc[28396](info): set up 2 sockets
Aug 11 17:21:07 dist-test-slave-mjjr krb5kdc[28396](info): commencing operation
krb5kdc: starting...
W20260811 17:21:08.846501 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:21:08 dist-test-slave-mjjr krb5kdc[28396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:21:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:21:08Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.157.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.157.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.157.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/rangeradmin_127.9.157.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.157.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/rangeradmin_127.9.157.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.157.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.157.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.157.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/rangerlookup_127.9.157.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.157.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/rangerlookup_127.9.157.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.157.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.157.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.157.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/HTTP_127.9.157.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.157.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/HTTP_127.9.157.148@KRBTEST.COM.keytab.
I20260811 17:21:09.049289 9846 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasklEMhqg/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/postgres -l logfile start
2026-08-11 17:21:11.687 UTC [28437] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 17:21:11.687 UTC [28437] LOG: listening on IPv4 address "127.9.157.148", port 41303
2026-08-11 17:21:11.691 UTC [28437] LOG: listening on Unix socket "/tmp/.s.PGSQL.41303"
2026-08-11 17:21:11.714 UTC [28444] LOG: database system was shut down at 2026-08-11 17:21:10 UTC
2026-08-11 17:21:11.718 UTC [28437] LOG: database system is ready to accept connections
I20260811 17:21:12.901234 9846 mini_postgres.cc:96] Postgres bound to 41303
2026-08-11 17:21:12.910 UTC [28461] FATAL: database "slave" does not exist
127.9.157.148:41303 - accepting connections
I20260811 17:21:12.910804 9846 mini_ranger.cc:161] Starting Ranger...
I20260811 17:21:12.929733 9846 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 17:21:13.470707 9846 mini_ranger.cc:87] Created ranger Postgres database
I20260811 17:21:13.470841 9846 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ranger-admin
2026-08-11 17:21:13,775 [I] DB FLAVOR :POSTGRES
2026-08-11 17:21:13,776 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 17:21:13,776 [I] Checking connection..
2026-08-11 17:21:13,776 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:41303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 17:21:14,067 [I] Checking connection passed.
2026-08-11 17:21:14,067 [I] --------- Verifying version history table ---------
2026-08-11 17:21:14,068 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:41303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 17:21:14,340 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 17:21:14,340 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:41303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 17:21:14,627 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 17:21:14,627 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 17:21:14,628 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:41303/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 17:21:14.909 UTC [28553] WARNING: there is no transaction in progress
2026-08-11 17:21:14,919 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 17:21:14,920 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:41303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 17:21:15,220 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 17:21:15,220 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 17:21:15,220 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:41303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 17:21:15,501 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:41303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 17:21:15,770 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:41303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-mjjr.c.gcp-upstream.internal','N') ;"
2026-08-11 17:21:16,055 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:41303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 17:21:16,361 [I] Table x_portal_user does not exist in database ranger
2026-08-11 17:21:16,361 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:41303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 17:21:16,624 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 17:21:16,624 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:41303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 17:21:16,892 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:41303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 17:21:17,165 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 17:21:17,165 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:41303/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 17:21:18.232 UTC [28721] WARNING: there is no transaction in progress
2026-08-11 17:21:18.251 UTC [28721] WARNING: there is no transaction in progress
2026-08-11 17:21:18.269 UTC [28721] WARNING: there is no transaction in progress
2026-08-11 17:21:18.287 UTC [28721] WARNING: there is no transaction in progress
2026-08-11 17:21:18.305 UTC [28721] WARNING: there is no transaction in progress
2026-08-11 17:21:18.328 UTC [28721] WARNING: there is no transaction in progress
2026-08-11 17:21:18.392 UTC [28721] WARNING: there is no transaction in progress
2026-08-11 17:21:18.399 UTC [28721] WARNING: there is no transaction in progress
2026-08-11 17:21:18.415 UTC [28721] WARNING: there is no transaction in progress
2026-08-11 17:21:18.426 UTC [28721] WARNING: there is no transaction in progress
2026-08-11 17:21:18.437 UTC [28721] WARNING: there is no transaction in progress
2026-08-11 17:21:18.452 UTC [28721] WARNING: there is no transaction in progress
2026-08-11 17:21:18.462 UTC [28721] WARNING: there is no transaction in progress
2026-08-11 17:21:18.469 UTC [28721] WARNING: there is no transaction in progress
2026-08-11 17:21:18.475 UTC [28721] WARNING: there is no transaction in progress
2026-08-11 17:21:18.482 UTC [28721] WARNING: there is no transaction in progress
2026-08-11 17:21:19,220 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 17:21:19,220 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:41303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 17:21:19,498 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 17:21:19,499 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:41303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 17:21:19,765 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 17:21:19,765 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:41303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 17:21:20,047 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 17:21:20,048 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:41303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 17:21:20,315 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:41303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 17:21:20,582 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:41303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-mjjr.c.gcp-upstream.internal';"
2026-08-11 17:21:20,831 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 17:21:20,831 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:41303/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 17:21:21,094 [I] DB_PATCHES have already been applied
I20260811 17:21:21.100284 9846 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ranger-admin:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasklEMhqg/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 17:21:21.100376 9846 mini_ranger.cc:193] Using host: 127.9.157.148
I20260811 17:21:21.105545 9846 mini_ranger.cc:227] Ranger admin URL: http://127.9.157.148:37231
Aug 11, 2026 5:21:21 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 5:21:21 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 5:21:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 5:21:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 5:21:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 5:21:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 5:21:22 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 17:21:22.507014 9846 mini_ranger.cc:160] Time spent starting Ranger: real 9.596s user 0.002s sys 0.013s
Aug 11, 2026 5:21:44 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 5:21:44 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 11, 2026 5:21:44 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 5:21:44 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 5:21:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 5:21:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 5:21:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 5:21:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 5:21:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 5:21:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 5:21:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 5:21:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 5:21:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 5:21:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 5:21:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 5:21:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 5:21:44 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 5:21:45 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 17:21:46.591482 9846 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.157.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.157.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.157.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/rangerkms_127.9.157.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.157.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/rangerkms_127.9.157.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.157.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.157.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.157.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/HTTP_127.9.157.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.157.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/HTTP_127.9.157.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 17:21:46 dist-test-slave-mjjr krb5kdc[28396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468906, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 17:21:46.708526 9846 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 17:21:46.722501 9846 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 17:21:47.632803 9846 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 17:21:47.632910 9846 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ranger-kms
I20260811 17:21:47.632934 9846 mini_ranger_kms.cc:227] Using postgres at 127.9.157.148:41303
2026-08-11 17:21:47,834 [I] DB FLAVOR :POSTGRES
2026-08-11 17:21:47,835 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 17:21:47,835 [I] Checking connection
2026-08-11 17:21:47,835 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:41303/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 17:21:48,136 [I] connection success
2026-08-11 17:21:48,136 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 17:21:48,136 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 17:21:48,136 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:41303/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 17:21:48,429 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 17:21:48,430 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 17:21:48,430 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 17:21:48,430 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.157.148:41303/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 17:21:48,718 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 17:21:49.278870 9846 mini_ranger_kms.cc:310] Created kms service
I20260811 17:21:49.519237 9846 mini_ranger_kms.cc:326] Created kudu user
I20260811 17:21:49.567597 9846 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 17:21:49 dist-test-slave-mjjr krb5kdc[28396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468906, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.157.148@KRBTEST.COM
I20260811 17:21:50.013399 9846 mini_ranger_kms.cc:384] Added ranger policy
I20260811 17:21:50.013579 9846 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ranger-kms:/tmp/dist-test-tasklEMhqg/build/debug/bin/postgresql.jar:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasklEMhqg/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 17:21:50.013603 9846 mini_ranger_kms.cc:240] Using host: 127.9.157.148
I20260811 17:21:50.018641 9846 mini_ranger_kms.cc:276] Ranger KMS PID: 28965
I20260811 17:21:50.018744 9846 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.157.148:56467
Aug 11, 2026 5:21:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 5:21:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 5:21:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 5:21:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 5:21:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 5:21:51 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 17:21:52 dist-test-slave-mjjr krb5kdc[28396](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786468912, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.157.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 17:21:52 dist-test-slave-mjjr krb5kdc[28396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468912, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.157.148@KRBTEST.COM for HTTP/127.9.157.148@KRBTEST.COM
I20260811 17:21:52.872169 9846 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.164s user 0.001s sys 0.182s
I20260811 17:21:52.872309 9846 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 17:21:52.872381 9846 mini_ranger_kms.cc:401] 127.9.157.148:56467/kms/v1/keys
Aug 11 17:21:53 dist-test-slave-mjjr krb5kdc[28396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468912, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.157.148@KRBTEST.COM for HTTP/127.9.157.148@KRBTEST.COM
Aug 11 17:21:55 dist-test-slave-mjjr krb5kdc[28396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:21:55.587570 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:37477
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:37477
--ranger_config_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.157.148:56467/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:21:55.757512 29035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:21:55.757823 29035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:21:55.757866 29035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:21:55.762006 29035 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:21:55.762084 29035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:21:55.762104 29035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:21:55.762120 29035 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:21:55.762135 29035 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:21:55.767964 29035 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.157.148:56467/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:37477
--ranger_config_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:37477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:21:55.769160 29035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:21:55.770233 29035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:21:55.780169 29035 server_base.cc:1034] running on GCE node
W20260811 17:21:55.780102 29046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:21:55.780128 29044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:21:55.780102 29042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:21:55.780987 29035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:21:55.782193 29035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:21:55.783382 29035 hybrid_clock.cc:648] HybridClock initialized: now 1786468915783362 us; error 34 us; skew 500 ppm
Aug 11 17:21:55 dist-test-slave-mjjr krb5kdc[28396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468915, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:21:55.786696 29035 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:21:55.787993 29035 webserver.cc:460] Webserver started at http://127.9.157.190:33129/ using document root <none> and password file <none>
I20260811 17:21:55.788630 29035 fs_manager.cc:359] Metadata directory not provided
I20260811 17:21:55.788713 29035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:21:55.788968 29035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 17:21:55 dist-test-slave-mjjr krb5kdc[28396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468915, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for HTTP/127.9.157.148@KRBTEST.COM
I20260811 17:21:55.856841 29035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "ce461148e94f403dac04a883809f14ba"
format_stamp: "Formatted at 2026-08-11 17:21:55 on dist-test-slave-mjjr"
server_key: "b3310da1bfc59a4ecb0d40b288dd1bd1"
server_key_iv: "6752a309bae2487e64690a4b6d86422c"
server_key_version: "kuduclusterkey@0"
I20260811 17:21:55.857570 29035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "ce461148e94f403dac04a883809f14ba"
format_stamp: "Formatted at 2026-08-11 17:21:55 on dist-test-slave-mjjr"
server_key: "b3310da1bfc59a4ecb0d40b288dd1bd1"
server_key_iv: "6752a309bae2487e64690a4b6d86422c"
server_key_version: "kuduclusterkey@0"
I20260811 17:21:55.862217 29035 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:21:55.884106 29055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:21:55.886090 29035 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260811 17:21:55.886298 29035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "ce461148e94f403dac04a883809f14ba"
format_stamp: "Formatted at 2026-08-11 17:21:55 on dist-test-slave-mjjr"
server_key: "b3310da1bfc59a4ecb0d40b288dd1bd1"
server_key_iv: "6752a309bae2487e64690a4b6d86422c"
server_key_version: "kuduclusterkey@0"
I20260811 17:21:55.886487 29035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:21:55.911865 29035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:21:55.915585 29035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:21:55.915903 29035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:21:55.926344 29035 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:37477
I20260811 17:21:55.926352 29130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:37477 every 8 connection(s)
I20260811 17:21:55.927913 29035 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:21:55.931010 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 29035
I20260811 17:21:55.931169 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:21:55.931455 29131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:21:55.931517 9846 external_mini_cluster.cc:1428] Setting key 991b278b95efb064e1276a98a2f731fb
I20260811 17:21:55.939392 29131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba: Bootstrap starting.
I20260811 17:21:55.942339 29131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba: Neither blocks nor log segments found. Creating new log.
I20260811 17:21:55.943480 29131 log.cc:824] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba: Log is configured to *not* fsync() on all Append() calls
Aug 11 17:21:55 dist-test-slave-mjjr krb5kdc[28396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:21:55.947202 29131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba: No bootstrap required, opened a new log
I20260811 17:21:55.951691 29131 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce461148e94f403dac04a883809f14ba" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37477 } }
I20260811 17:21:55.951985 29137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:55.933129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50042 (local address 127.9.157.190:37477)
0811 17:21:55.933682 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0811 17:21:55.933703 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 17:21:55.933893 (+ 190us) server_negotiation.cc:407] Connection header received
0811 17:21:55.934893 (+ 1000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:21:55.934920 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:21:55.935426 (+ 506us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:21:55.935860 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:21:55.937411 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:55.938916 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:55.940269 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:55.940990 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:55.941161 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:55.945108 (+ 3947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:21:55.945115 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:21:55.945165 (+ 50us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:21:55.945196 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:21:55.948260 (+ 3064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:21:55.948912 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:21:55.948920 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:21:55.948929 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:21:55.949081 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:21:55.949434 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:21:55.949437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:21:55.949439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:21:55.950026 (+ 587us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:21:55.950242 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 17:21:55.950573 (+ 331us) server_negotiation.cc:299] Negotiation successful
0811 17:21:55.950910 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":147,"threads_started":1}
I20260811 17:21:55.952019 29131 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:21:55.952081 29131 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce461148e94f403dac04a883809f14ba, State: Initialized, Role: FOLLOWER
I20260811 17:21:55.952621 29131 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce461148e94f403dac04a883809f14ba" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37477 } }
I20260811 17:21:55.952801 29131 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:21:55.952898 29131 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:21:55.953009 29131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:21:55.954093 29131 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce461148e94f403dac04a883809f14ba" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37477 } }
I20260811 17:21:55.954512 29131 leader_election.cc:302] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce461148e94f403dac04a883809f14ba; no voters:
I20260811 17:21:55.954969 29131 leader_election.cc:288] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:21:55.955051 29139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:21:55.955265 29139 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [term 1 LEADER]: Becoming Leader. State: Replica: ce461148e94f403dac04a883809f14ba, State: Running, Role: LEADER
I20260811 17:21:55.955667 29139 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce461148e94f403dac04a883809f14ba" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37477 } }
I20260811 17:21:55.956442 29131 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:21:55.957528 29145 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce461148e94f403dac04a883809f14ba. Latest consensus state: current_term: 1 leader_uuid: "ce461148e94f403dac04a883809f14ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce461148e94f403dac04a883809f14ba" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37477 } } }
I20260811 17:21:55.957670 29145 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [sys.catalog]: This master's current role is: LEADER
I20260811 17:21:55.958102 29154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:21:55.958115 29143 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce461148e94f403dac04a883809f14ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce461148e94f403dac04a883809f14ba" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37477 } } }
I20260811 17:21:55.958251 29143 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [sys.catalog]: This master's current role is: LEADER
I20260811 17:21:55.961824 29154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:21:55.962378 29131 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 17:21:55.973632 29154 catalog_manager.cc:1347] Generated new cluster ID: 3f0db458658d4313ada1b48c34be7bae
I20260811 17:21:55.973869 29154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:21:56.003926 29154 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:21:56.005555 29154 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:21:56.015398 29154 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba: Generated new TSK 0
I20260811 17:21:56.016639 29154 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
I20260811 17:21:56.743638 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.157.148:56467/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:0
--local_ip_for_outbound_sockets=127.9.157.129
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:37477
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:21:56.898555 29203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:21:56.899003 29203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:21:56.899075 29203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:21:56.905635 29203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:21:56.905751 29203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:21:56.905958 29203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:21:56.914537 29203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.157.148:56467/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:37477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:21:56.915953 29203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:21:56.917501 29203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:21:56.931236 29214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:21:56.931520 29213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:21:56.932032 29203 server_base.cc:1034] running on GCE node
W20260811 17:21:56.933882 29217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:21:56.934551 29203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:21:56.935472 29203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:21:56.936663 29203 hybrid_clock.cc:648] HybridClock initialized: now 1786468916936615 us; error 73 us; skew 500 ppm
Aug 11 17:21:56 dist-test-slave-mjjr krb5kdc[28396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468916, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:21:56.939877 29203 init.cc:377] Logged in from keytab as kudu/127.9.157.129@KRBTEST.COM (short username kudu)
I20260811 17:21:56.940887 29203 webserver.cc:460] Webserver started at http://127.9.157.129:35297/ using document root <none> and password file <none>
I20260811 17:21:56.941537 29203 fs_manager.cc:359] Metadata directory not provided
I20260811 17:21:56.941637 29203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:21:56.941890 29203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 17:21:56 dist-test-slave-mjjr krb5kdc[28396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468916, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for HTTP/127.9.157.148@KRBTEST.COM
I20260811 17:21:56.976048 29203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/data/instance:
uuid: "c2a9d4a3347f4ebb95194e514cda12be"
format_stamp: "Formatted at 2026-08-11 17:21:56 on dist-test-slave-mjjr"
server_key: "55ea611189e3eb5936d48bd22cbde31f"
server_key_iv: "0cc89f14eb55bfa25bb300d3da49db73"
server_key_version: "kuduclusterkey@0"
I20260811 17:21:56.976985 29203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance:
uuid: "c2a9d4a3347f4ebb95194e514cda12be"
format_stamp: "Formatted at 2026-08-11 17:21:56 on dist-test-slave-mjjr"
server_key: "55ea611189e3eb5936d48bd22cbde31f"
server_key_iv: "0cc89f14eb55bfa25bb300d3da49db73"
server_key_version: "kuduclusterkey@0"
I20260811 17:21:56.984054 29203 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 17:21:57.002915 29228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:21:57.004804 29203 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 17:21:57.005055 29203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "c2a9d4a3347f4ebb95194e514cda12be"
format_stamp: "Formatted at 2026-08-11 17:21:56 on dist-test-slave-mjjr"
server_key: "55ea611189e3eb5936d48bd22cbde31f"
server_key_iv: "0cc89f14eb55bfa25bb300d3da49db73"
server_key_version: "kuduclusterkey@0"
I20260811 17:21:57.005308 29203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:21:57.032002 29203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:21:57.036116 29203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:21:57.036363 29203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:21:57.037061 29203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:21:57.038089 29203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:21:57.038158 29203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:21:57.038256 29203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:21:57.038305 29203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:21:57.050602 29203 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:40103
I20260811 17:21:57.051156 29387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:40103 every 8 connection(s)
I20260811 17:21:57.052127 29203 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:21:57.054728 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 29203
I20260811 17:21:57.054869 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance
I20260811 17:21:57.055188 9846 external_mini_cluster.cc:1428] Setting key 7fc04b3ba3c9c1731cfea1f80697c935
Aug 11 17:21:57 dist-test-slave-mjjr krb5kdc[28396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468916, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.157.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.130@KRBTEST.COM" created.
Aug 11 17:21:57 dist-test-slave-mjjr krb5kdc[28396](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786468917, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:21:57.080184 29393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:57.056477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:53609 (local address 127.9.157.190:37477)
0811 17:21:57.063102 (+ 6625us) server_negotiation.cc:206] Beginning negotiation
0811 17:21:57.063108 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:21:57.063125 (+ 17us) server_negotiation.cc:407] Connection header received
0811 17:21:57.063175 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:21:57.063178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:21:57.063235 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:21:57.063337 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:21:57.066101 (+ 2764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:57.067037 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:57.069091 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:57.069511 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:57.069643 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:57.073704 (+ 4061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:21:57.073710 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:21:57.073713 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:21:57.073751 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:21:57.075940 (+ 2189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:21:57.076807 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:21:57.076810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:21:57.076813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:21:57.076929 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:21:57.077473 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:21:57.077477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:21:57.077479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:21:57.077759 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:21:57.077869 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 17:21:57.079826 (+ 1957us) server_negotiation.cc:299] Negotiation successful
0811 17:21:57.079990 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6513,"thread_start_us":77,"threads_started":1}
I20260811 17:21:57.081382 29392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:57.055118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:37477 (local address 127.9.157.129:53609)
0811 17:21:57.060207 (+ 5089us) negotiation.cc:107] Waiting for socket to connect
0811 17:21:57.060307 (+ 100us) client_negotiation.cc:174] Beginning negotiation
0811 17:21:57.062109 (+ 1802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:21:57.063785 (+ 1676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:21:57.063798 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:21:57.064506 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:21:57.065816 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:57.065845 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:57.067247 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:57.067251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:57.068876 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:57.068894 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:57.070257 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:57.070267 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:57.070533 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:57.071338 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:21:57.071364 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:21:57.073400 (+ 2036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:21:57.076150 (+ 2750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:21:57.076160 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:21:57.076185 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:21:57.076644 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:21:57.077052 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:21:57.077056 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:21:57.077058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:21:57.077296 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:21:57.077940 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:21:57.077948 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:21:57.078393 (+ 445us) client_negotiation.cc:769] Sending connection context
0811 17:21:57.078652 (+ 259us) client_negotiation.cc:240] Negotiation successful
0811 17:21:57.079086 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1839,"thread_start_us":189,"threads_started":1}
I20260811 17:21:57.083338 29389 heartbeater.cc:347] Connected to a master server at 127.9.157.190:37477
I20260811 17:21:57.083633 29389 heartbeater.cc:464] Registering TS with master...
I20260811 17:21:57.084367 29389 heartbeater.cc:511] Master 127.9.157.190:37477 requested a full tablet report, sending...
I20260811 17:21:57.086490 29083 ts_manager.cc:194] Registered new tserver with Master: c2a9d4a3347f4ebb95194e514cda12be (127.9.157.129:40103)
I20260811 17:21:57.088054 29083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.129@KRBTEST.COM'} at 127.9.157.129:53609
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
I20260811 17:21:57.103013 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.157.148:56467/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:0
--local_ip_for_outbound_sockets=127.9.157.130
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:37477
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:21:57.241986 29398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:21:57.242327 29398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:21:57.242445 29398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:21:57.247275 29398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:21:57.247457 29398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:21:57.247602 29398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:21:57.255860 29398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.157.148:56467/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:37477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:21:57.257413 29398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:21:57.258917 29398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:21:57.271931 29409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:21:57.272732 29406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:21:57.272732 29407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:21:57.274228 29398 server_base.cc:1034] running on GCE node
I20260811 17:21:57.274950 29398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:21:57.275786 29398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:21:57.276988 29398 hybrid_clock.cc:648] HybridClock initialized: now 1786468917276949 us; error 52 us; skew 500 ppm
Aug 11 17:21:57 dist-test-slave-mjjr krb5kdc[28396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468917, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:21:57.280105 29398 init.cc:377] Logged in from keytab as kudu/127.9.157.130@KRBTEST.COM (short username kudu)
I20260811 17:21:57.281232 29398 webserver.cc:460] Webserver started at http://127.9.157.130:37873/ using document root <none> and password file <none>
I20260811 17:21:57.281859 29398 fs_manager.cc:359] Metadata directory not provided
I20260811 17:21:57.281953 29398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:21:57.282186 29398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 17:21:57 dist-test-slave-mjjr krb5kdc[28396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468917, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for HTTP/127.9.157.148@KRBTEST.COM
I20260811 17:21:57.298921 29398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/data/instance:
uuid: "31b3f97b137246a9bfb86e6d4a79258a"
format_stamp: "Formatted at 2026-08-11 17:21:57 on dist-test-slave-mjjr"
server_key: "e287c629b31aad32883b4d3e42b82964"
server_key_iv: "18073166af63301aeb787362d822cc4c"
server_key_version: "kuduclusterkey@0"
I20260811 17:21:57.299759 29398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance:
uuid: "31b3f97b137246a9bfb86e6d4a79258a"
format_stamp: "Formatted at 2026-08-11 17:21:57 on dist-test-slave-mjjr"
server_key: "e287c629b31aad32883b4d3e42b82964"
server_key_iv: "18073166af63301aeb787362d822cc4c"
server_key_version: "kuduclusterkey@0"
I20260811 17:21:57.313905 29398 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.005s sys 0.000s
I20260811 17:21:57.334134 29420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:21:57.336165 29398 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 17:21:57.336342 29398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "31b3f97b137246a9bfb86e6d4a79258a"
format_stamp: "Formatted at 2026-08-11 17:21:57 on dist-test-slave-mjjr"
server_key: "e287c629b31aad32883b4d3e42b82964"
server_key_iv: "18073166af63301aeb787362d822cc4c"
server_key_version: "kuduclusterkey@0"
I20260811 17:21:57.336503 29398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:21:57.358764 29398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:21:57.362571 29398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:21:57.362808 29398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:21:57.363703 29398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:21:57.364859 29398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:21:57.364974 29398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:21:57.365075 29398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:21:57.365136 29398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:21:57.379923 29398 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:41831
I20260811 17:21:57.380018 29583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:41831 every 8 connection(s)
I20260811 17:21:57.380972 29398 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:21:57.390059 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 29398
I20260811 17:21:57.390206 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance
I20260811 17:21:57.390548 9846 external_mini_cluster.cc:1428] Setting key c8adec0399308718a21167146892034e
Aug 11 17:21:57 dist-test-slave-mjjr krb5kdc[28396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468917, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:21:57.397395 29393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:57.383012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:39537 (local address 127.9.157.190:37477)
0811 17:21:57.383177 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 17:21:57.383182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:21:57.384162 (+ 980us) server_negotiation.cc:407] Connection header received
0811 17:21:57.385221 (+ 1059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:21:57.385225 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:21:57.385288 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:21:57.385407 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:21:57.386756 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:57.387316 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:57.388650 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:57.388960 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:57.389132 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:57.392335 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:21:57.392339 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:21:57.392342 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:21:57.392380 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:21:57.394276 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:21:57.395038 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:21:57.395043 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:21:57.395047 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:21:57.395153 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:21:57.395615 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:21:57.395620 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:21:57.395623 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:21:57.395949 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:21:57.396065 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 17:21:57.396956 (+ 891us) server_negotiation.cc:299] Negotiation successful
0811 17:21:57.397139 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 17:21:57.398840 29586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:57.383410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:37477 (local address 127.9.157.130:39537)
0811 17:21:57.383995 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0811 17:21:57.384050 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 17:21:57.384992 (+ 942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:21:57.385598 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:21:57.385605 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:21:57.385913 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:21:57.386587 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:57.386605 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:57.387478 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:57.387482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:57.388484 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:57.388496 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:57.389180 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:57.389185 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:57.389396 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:57.390145 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:21:57.390173 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:21:57.392072 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:21:57.394431 (+ 2359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:21:57.394441 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:21:57.394462 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:21:57.394877 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:21:57.395257 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:21:57.395260 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:21:57.395262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:21:57.395488 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:21:57.396080 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:21:57.396090 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:21:57.396523 (+ 433us) client_negotiation.cc:769] Sending connection context
0811 17:21:57.396847 (+ 324us) client_negotiation.cc:240] Negotiation successful
0811 17:21:57.397318 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":204,"threads_started":1}
I20260811 17:21:57.400447 29584 heartbeater.cc:347] Connected to a master server at 127.9.157.190:37477
I20260811 17:21:57.400789 29584 heartbeater.cc:464] Registering TS with master...
I20260811 17:21:57.401580 29584 heartbeater.cc:511] Master 127.9.157.190:37477 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.157.131@KRBTEST.COM; defaulting to no policy
I20260811 17:21:57.403120 29083 ts_manager.cc:194] Registered new tserver with Master: 31b3f97b137246a9bfb86e6d4a79258a (127.9.157.130:41831)
I20260811 17:21:57.403911 29083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.130@KRBTEST.COM'} at 127.9.157.130:39537
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
I20260811 17:21:57.433137 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.157.148:56467/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:0
--local_ip_for_outbound_sockets=127.9.157.131
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:37477
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 17:21:57 dist-test-slave-mjjr krb5kdc[28396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468917, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for HTTP/127.9.157.148@KRBTEST.COM
W20260811 17:21:57.615119 29589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:21:57.615758 29589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:21:57.615899 29589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:21:57.622313 29589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:21:57.622573 29589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:21:57.622714 29589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:21:57.631000 29589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.157.148:56467/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:37477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:21:57.633060 29589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:21:57.634618 29589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:21:57.648823 29600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:21:57.649886 29602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:21:57.650246 29589 server_base.cc:1034] running on GCE node
W20260811 17:21:57.650467 29599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:21:57.651036 29589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:21:57.651907 29589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:21:57.653136 29589 hybrid_clock.cc:648] HybridClock initialized: now 1786468917653065 us; error 86 us; skew 500 ppm
Aug 11 17:21:57 dist-test-slave-mjjr krb5kdc[28396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468917, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:21:57.656983 29589 init.cc:377] Logged in from keytab as kudu/127.9.157.131@KRBTEST.COM (short username kudu)
I20260811 17:21:57.658221 29589 webserver.cc:460] Webserver started at http://127.9.157.131:32951/ using document root <none> and password file <none>
I20260811 17:21:57.659024 29589 fs_manager.cc:359] Metadata directory not provided
I20260811 17:21:57.659168 29589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:21:57.659469 29589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 17:21:57 dist-test-slave-mjjr krb5kdc[28396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468917, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for HTTP/127.9.157.148@KRBTEST.COM
I20260811 17:21:57.690429 29589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/data/instance:
uuid: "ffc41ec094fc41ba81ce936bfeb92cec"
format_stamp: "Formatted at 2026-08-11 17:21:57 on dist-test-slave-mjjr"
server_key: "f8dd7fed6695bb1e95bcc8411480b5ed"
server_key_iv: "671a3b876258983e385d43caa4afab49"
server_key_version: "kuduclusterkey@0"
I20260811 17:21:57.691227 29589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance:
uuid: "ffc41ec094fc41ba81ce936bfeb92cec"
format_stamp: "Formatted at 2026-08-11 17:21:57 on dist-test-slave-mjjr"
server_key: "f8dd7fed6695bb1e95bcc8411480b5ed"
server_key_iv: "671a3b876258983e385d43caa4afab49"
server_key_version: "kuduclusterkey@0"
I20260811 17:21:57.697278 29589 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 17:21:57.715556 29612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:21:57.717474 29589 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 17:21:57.717623 29589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "ffc41ec094fc41ba81ce936bfeb92cec"
format_stamp: "Formatted at 2026-08-11 17:21:57 on dist-test-slave-mjjr"
server_key: "f8dd7fed6695bb1e95bcc8411480b5ed"
server_key_iv: "671a3b876258983e385d43caa4afab49"
server_key_version: "kuduclusterkey@0"
I20260811 17:21:57.717861 29589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:21:57.738554 29589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:21:57.742211 29589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:21:57.742575 29589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:21:57.743844 29589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:21:57.744948 29589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:21:57.745023 29589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:21:57.745126 29589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:21:57.745172 29589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:21:57.758680 29589 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:44923
I20260811 17:21:57.758721 29780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:44923 every 8 connection(s)
I20260811 17:21:57.760478 29589 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:21:57.761298 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 29589
I20260811 17:21:57.761435 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance
I20260811 17:21:57.761729 9846 external_mini_cluster.cc:1428] Setting key d2f755c74cbf9134bf96e26b3eaa9fc7
Aug 11 17:21:57 dist-test-slave-mjjr krb5kdc[28396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468917, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:21:57.778657 29393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:57.764643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:35377 (local address 127.9.157.190:37477)
0811 17:21:57.764763 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 17:21:57.764768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:21:57.764871 (+ 103us) server_negotiation.cc:407] Connection header received
0811 17:21:57.766142 (+ 1271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:21:57.766146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:21:57.766203 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:21:57.766311 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:21:57.767809 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:57.768642 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:57.770275 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:57.770625 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:57.770740 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:57.773936 (+ 3196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:21:57.773940 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:21:57.773943 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:21:57.773973 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:21:57.775866 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:21:57.776597 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:21:57.776601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:21:57.776603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:21:57.776706 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:21:57.777160 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:21:57.777164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:21:57.777166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:21:57.777443 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:21:57.777558 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 17:21:57.778306 (+ 748us) server_negotiation.cc:299] Negotiation successful
0811 17:21:57.778469 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 17:21:57.780262 29783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:57.763458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:37477 (local address 127.9.157.131:35377)
0811 17:21:57.764692 (+ 1234us) negotiation.cc:107] Waiting for socket to connect
0811 17:21:57.764752 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 17:21:57.765879 (+ 1127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:21:57.766538 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:21:57.766550 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:21:57.766920 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:21:57.767619 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:57.767665 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:57.768803 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:57.768807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:57.770135 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:57.770147 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:57.770807 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:57.770811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:57.771015 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:57.771808 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:21:57.771837 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:21:57.773702 (+ 1865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:21:57.776019 (+ 2317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:21:57.776026 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:21:57.776046 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:21:57.776458 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:21:57.776819 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:21:57.776823 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:21:57.776825 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:21:57.777029 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:21:57.777575 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:21:57.777583 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:21:57.777937 (+ 354us) client_negotiation.cc:769] Sending connection context
0811 17:21:57.778219 (+ 282us) client_negotiation.cc:240] Negotiation successful
0811 17:21:57.778963 (+ 744us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":974,"thread_start_us":196,"threads_started":1}
I20260811 17:21:57.782015 29781 heartbeater.cc:347] Connected to a master server at 127.9.157.190:37477
I20260811 17:21:57.782333 29781 heartbeater.cc:464] Registering TS with master...
I20260811 17:21:57.782982 29781 heartbeater.cc:511] Master 127.9.157.190:37477 requested a full tablet report, sending...
I20260811 17:21:57.784452 29083 ts_manager.cc:194] Registered new tserver with Master: ffc41ec094fc41ba81ce936bfeb92cec (127.9.157.131:44923)
I20260811 17:21:57.785279 29083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.131@KRBTEST.COM'} at 127.9.157.131:35377
I20260811 17:21:57.789909 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:21:57.790764 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 29203
I20260811 17:21:57.818928 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 29398
I20260811 17:21:57.875814 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 29589
I20260811 17:21:57.891783 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 29035
I20260811 17:21:57.906675 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:37477
--webserver_interface=127.9.157.190
--webserver_port=33129
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:37477
--ranger_config_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.157.148:56467/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.157.148:56467/kms with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:21:58.035147 29784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:21:58.035502 29784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:21:58.035573 29784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:21:58.040504 29784 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:21:58.040612 29784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:21:58.040634 29784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:21:58.040652 29784 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:21:58.040666 29784 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:21:58.046954 29784 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.157.148:56467/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:37477
--ranger_config_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:37477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=33129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:21:58.048342 29784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:21:58.049708 29784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:21:58.060071 29795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:21:58.060328 29784 server_base.cc:1034] running on GCE node
W20260811 17:21:58.060217 29792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:21:58.060071 29793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:21:58.061331 29784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:21:58.062711 29784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:21:58.063973 29784 hybrid_clock.cc:648] HybridClock initialized: now 1786468918063942 us; error 59 us; skew 500 ppm
Aug 11 17:21:58 dist-test-slave-mjjr krb5kdc[28396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468918, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:21:58.067502 29784 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:21:58.068761 29784 webserver.cc:460] Webserver started at http://127.9.157.190:33129/ using document root <none> and password file <none>
I20260811 17:21:58.069496 29784 fs_manager.cc:359] Metadata directory not provided
I20260811 17:21:58.069613 29784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 17:21:58 dist-test-slave-mjjr krb5kdc[28396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468918, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for HTTP/127.9.157.148@KRBTEST.COM
I20260811 17:21:58.161386 29784 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 17:21:58.164397 29806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:21:58.166355 29784 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:21:58.166561 29784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "ce461148e94f403dac04a883809f14ba"
format_stamp: "Formatted at 2026-08-11 17:21:55 on dist-test-slave-mjjr"
server_key: "b3310da1bfc59a4ecb0d40b288dd1bd1"
server_key_iv: "6752a309bae2487e64690a4b6d86422c"
server_key_version: "kuduclusterkey@0"
I20260811 17:21:58.167300 29784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:21:58.194442 29784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:21:58.198784 29784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:21:58.198995 29784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:21:58.207432 29784 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:37477
I20260811 17:21:58.207446 29876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:37477 every 8 connection(s)
I20260811 17:21:58.209321 29784 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:21:58.212852 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 29784
I20260811 17:21:58.213882 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.157.148:56467/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:40103
--local_ip_for_outbound_sockets=127.9.157.129
--tserver_master_addrs=127.9.157.190:37477
--webserver_port=35297
--webserver_interface=127.9.157.129
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.157.148:56467/kms with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:21:58.215206 29878 sys_catalog.cc:263] Verifying existing consensus state
I20260811 17:21:58.217417 29878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba: Bootstrap starting.
I20260811 17:21:58.236150 29878 log.cc:824] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba: Log is configured to *not* fsync() on all Append() calls
I20260811 17:21:58.246145 29878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 17:21:58.246805 29878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba: Bootstrap complete.
I20260811 17:21:58.252602 29878 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce461148e94f403dac04a883809f14ba" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37477 } }
I20260811 17:21:58.253329 29878 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce461148e94f403dac04a883809f14ba, State: Initialized, Role: FOLLOWER
I20260811 17:21:58.254258 29878 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce461148e94f403dac04a883809f14ba" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37477 } }
I20260811 17:21:58.254470 29878 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:21:58.254549 29878 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:21:58.254662 29878 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [term 1 FOLLOWER]: Advancing to term 2
I20260811 17:21:58.257160 29878 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce461148e94f403dac04a883809f14ba" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37477 } }
I20260811 17:21:58.257565 29878 leader_election.cc:302] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce461148e94f403dac04a883809f14ba; no voters:
I20260811 17:21:58.257966 29878 leader_election.cc:288] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 17:21:58.258119 29883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [term 2 FOLLOWER]: Leader election won for term 2
I20260811 17:21:58.258388 29883 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [term 2 LEADER]: Becoming Leader. State: Replica: ce461148e94f403dac04a883809f14ba, State: Running, Role: LEADER
I20260811 17:21:58.258744 29883 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce461148e94f403dac04a883809f14ba" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37477 } }
I20260811 17:21:58.259392 29878 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:21:58.260522 29885 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ce461148e94f403dac04a883809f14ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce461148e94f403dac04a883809f14ba" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37477 } } }
I20260811 17:21:58.260684 29885 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [sys.catalog]: This master's current role is: LEADER
I20260811 17:21:58.261003 29887 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce461148e94f403dac04a883809f14ba. Latest consensus state: current_term: 2 leader_uuid: "ce461148e94f403dac04a883809f14ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce461148e94f403dac04a883809f14ba" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37477 } } }
I20260811 17:21:58.261104 29887 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba [sys.catalog]: This master's current role is: LEADER
I20260811 17:21:58.261209 29898 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:21:58.265021 29898 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:21:58.265672 29878 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 17:21:58.266157 29898 catalog_manager.cc:1259] Loaded cluster ID: 3f0db458658d4313ada1b48c34be7bae
I20260811 17:21:58.266420 29898 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:21:58.269562 29898 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:21:58.270458 29898 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba: Loaded TSK: 0
I20260811 17:21:58.271896 29898 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 17:21:58.390771 29881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:21:58.391217 29881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:21:58.391287 29881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:21:58.398135 29881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:21:58.398265 29881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:21:58.398363 29881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:21:58.407727 29881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.157.148:56467/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:40103
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=35297
--tserver_master_addrs=127.9.157.190:37477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:21:58.409524 29881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:21:58.411147 29881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:21:58.426504 29934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:21:58.426913 29939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:21:58.426507 29933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:21:58.427255 29881 server_base.cc:1034] running on GCE node
I20260811 17:21:58.427850 29881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:21:58.428910 29881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:21:58.432269 29881 hybrid_clock.cc:648] HybridClock initialized: now 1786468918432158 us; error 149 us; skew 500 ppm
Aug 11 17:21:58 dist-test-slave-mjjr krb5kdc[28396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468918, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:21:58.436522 29881 init.cc:377] Logged in from keytab as kudu/127.9.157.129@KRBTEST.COM (short username kudu)
I20260811 17:21:58.437980 29881 webserver.cc:460] Webserver started at http://127.9.157.129:35297/ using document root <none> and password file <none>
I20260811 17:21:58.438829 29881 fs_manager.cc:359] Metadata directory not provided
I20260811 17:21:58.438964 29881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 17:21:58 dist-test-slave-mjjr krb5kdc[28396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468918, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for HTTP/127.9.157.148@KRBTEST.COM
I20260811 17:21:58.464253 29881 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 17:21:58.467398 29947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:21:58.469003 29881 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 17:21:58.469218 29881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "c2a9d4a3347f4ebb95194e514cda12be"
format_stamp: "Formatted at 2026-08-11 17:21:56 on dist-test-slave-mjjr"
server_key: "55ea611189e3eb5936d48bd22cbde31f"
server_key_iv: "0cc89f14eb55bfa25bb300d3da49db73"
server_key_version: "kuduclusterkey@0"
I20260811 17:21:58.469837 29881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:21:58.517280 29881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:21:58.525280 29881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:21:58.525557 29881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:21:58.526410 29881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:21:58.527709 29881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:21:58.527777 29881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:21:58.527850 29881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:21:58.527882 29881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:21:58.540064 29881 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:40103
I20260811 17:21:58.540835 30120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:40103 every 8 connection(s)
I20260811 17:21:58.541512 29881 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:21:58.547606 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 29881
I20260811 17:21:58.548224 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.157.148:56467/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:41831
--local_ip_for_outbound_sockets=127.9.157.130
--tserver_master_addrs=127.9.157.190:37477
--webserver_port=37873
--webserver_interface=127.9.157.130
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.157.148:56467/kms with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 17:21:58 dist-test-slave-mjjr krb5kdc[28396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468918, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:21:58.566277 30123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:58.544079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:49539 (local address 127.9.157.190:37477)
0811 17:21:58.546363 (+ 2284us) server_negotiation.cc:206] Beginning negotiation
0811 17:21:58.546380 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:21:58.546420 (+ 40us) server_negotiation.cc:407] Connection header received
0811 17:21:58.546752 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:21:58.546768 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:21:58.547140 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:21:58.547500 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:21:58.549929 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:58.551185 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:58.553113 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:58.553926 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:58.554103 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:58.558168 (+ 4065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:21:58.558180 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:21:58.558208 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:21:58.558258 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:21:58.560995 (+ 2737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:21:58.561941 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:21:58.561952 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:21:58.561962 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:21:58.562149 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:21:58.562684 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:21:58.562690 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:21:58.562695 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:21:58.563343 (+ 648us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:21:58.563491 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 17:21:58.564490 (+ 999us) server_negotiation.cc:299] Negotiation successful
0811 17:21:58.564917 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2111,"thread_start_us":106,"threads_started":1}
I20260811 17:21:58.568298 30124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:58.544228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:37477 (local address 127.9.157.129:49539)
0811 17:21:58.544793 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0811 17:21:58.544864 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 17:21:58.545831 (+ 967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:21:58.548227 (+ 2396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:21:58.548238 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:21:58.548921 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:21:58.549671 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:58.549697 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:58.551363 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:58.551370 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:58.552927 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:58.552946 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:58.554445 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:58.554452 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:58.554696 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:58.555710 (+ 1014us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:21:58.555740 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:21:58.557822 (+ 2082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:21:58.561255 (+ 3433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:21:58.561266 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:21:58.561288 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:21:58.561758 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:21:58.562297 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:21:58.562302 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:21:58.562304 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:21:58.562545 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:21:58.563580 (+ 1035us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:21:58.563590 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:21:58.564025 (+ 435us) client_negotiation.cc:769] Sending connection context
0811 17:21:58.566848 (+ 2823us) client_negotiation.cc:240] Negotiation successful
0811 17:21:58.567245 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":156,"threads_started":1}
I20260811 17:21:58.571288 30121 heartbeater.cc:347] Connected to a master server at 127.9.157.190:37477
I20260811 17:21:58.571594 30121 heartbeater.cc:464] Registering TS with master...
I20260811 17:21:58.572515 30121 heartbeater.cc:511] Master 127.9.157.190:37477 requested a full tablet report, sending...
W20260811 17:21:58.575335 30121 heartbeater.cc:644] Failed to heartbeat to 127.9.157.190:37477 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 17:21:58.575515 30121 heartbeater.cc:464] Registering TS with master...
I20260811 17:21:58.575783 30121 heartbeater.cc:511] Master 127.9.157.190:37477 requested a full tablet report, sending...
I20260811 17:21:58.576830 30121 heartbeater.cc:464] Registering TS with master...
I20260811 17:21:58.577056 30121 heartbeater.cc:511] Master 127.9.157.190:37477 requested a full tablet report, sending...
W20260811 17:21:58.577931 30121 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 17:21:58.711848 30126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:21:58.712211 30126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:21:58.712304 30126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:21:58.716893 30126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:21:58.717007 30126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:21:58.717108 30126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:21:58.723460 30126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.157.148:56467/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:41831
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=37873
--tserver_master_addrs=127.9.157.190:37477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:21:58.724828 30126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:21:58.726066 30126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:21:58.738229 30137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:21:58.738121 30139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:21:58.738147 30135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:21:58.738560 30126 server_base.cc:1034] running on GCE node
I20260811 17:21:58.739212 30126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:21:58.740202 30126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:21:58.741420 30126 hybrid_clock.cc:648] HybridClock initialized: now 1786468918741380 us; error 53 us; skew 500 ppm
Aug 11 17:21:58 dist-test-slave-mjjr krb5kdc[28396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468918, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:21:58.748383 30126 init.cc:377] Logged in from keytab as kudu/127.9.157.130@KRBTEST.COM (short username kudu)
I20260811 17:21:58.750010 30126 webserver.cc:460] Webserver started at http://127.9.157.130:37873/ using document root <none> and password file <none>
I20260811 17:21:58.750974 30126 fs_manager.cc:359] Metadata directory not provided
I20260811 17:21:58.751194 30126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 17:21:58 dist-test-slave-mjjr krb5kdc[28396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468918, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for HTTP/127.9.157.148@KRBTEST.COM
I20260811 17:21:58.784994 30126 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.002s sys 0.001s
I20260811 17:21:58.788906 30151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:21:58.790848 30126 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 17:21:58.791013 30126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "31b3f97b137246a9bfb86e6d4a79258a"
format_stamp: "Formatted at 2026-08-11 17:21:57 on dist-test-slave-mjjr"
server_key: "e287c629b31aad32883b4d3e42b82964"
server_key_iv: "18073166af63301aeb787362d822cc4c"
server_key_version: "kuduclusterkey@0"
I20260811 17:21:58.791847 30126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:21:58.835306 30126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:21:58.839720 30126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:21:58.840044 30126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:21:58.840945 30126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:21:58.842294 30126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:21:58.842414 30126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 17:21:58.842525 30126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:21:58.842589 30126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:21:58.860150 30126 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:41831
I20260811 17:21:58.860220 30325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:41831 every 8 connection(s)
I20260811 17:21:58.861816 30126 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:21:58.867756 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 30126
I20260811 17:21:58.868419 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.157.148:56467/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:44923
--local_ip_for_outbound_sockets=127.9.157.131
--tserver_master_addrs=127.9.157.190:37477
--webserver_port=32951
--webserver_interface=127.9.157.131
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.157.148:56467/kms with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 17:21:58 dist-test-slave-mjjr krb5kdc[28396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468918, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:21:58.886121 30123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:58.864461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:45147 (local address 127.9.157.190:37477)
0811 17:21:58.864628 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 17:21:58.864634 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:21:58.865701 (+ 1067us) server_negotiation.cc:407] Connection header received
0811 17:21:58.871681 (+ 5980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:21:58.871687 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:21:58.871779 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:21:58.871845 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:21:58.874113 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:58.875061 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:58.877451 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:58.877879 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:58.877981 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:58.881729 (+ 3748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:21:58.881733 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:21:58.881736 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:21:58.881769 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:21:58.883502 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:21:58.884241 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:21:58.884245 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:21:58.884246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:21:58.884332 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:21:58.884746 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:21:58.884751 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:21:58.884753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:21:58.884983 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:21:58.885101 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 17:21:58.885736 (+ 635us) server_negotiation.cc:299] Negotiation successful
0811 17:21:58.885916 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 17:21:58.887214 30328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:58.864910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:37477 (local address 127.9.157.130:45147)
0811 17:21:58.865505 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0811 17:21:58.865567 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 17:21:58.866739 (+ 1172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:21:58.872507 (+ 5768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:21:58.872521 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:21:58.872980 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:21:58.873849 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:58.873880 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:58.875283 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:58.875287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:58.877210 (+ 1923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:58.877230 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:58.878264 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:58.878268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:58.878535 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:58.879391 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:21:58.879418 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:21:58.881445 (+ 2027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:21:58.883721 (+ 2276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:21:58.883729 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:21:58.883758 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:21:58.884099 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:21:58.884441 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:21:58.884445 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:21:58.884447 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:21:58.884629 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:21:58.885120 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:21:58.885129 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:21:58.885452 (+ 323us) client_negotiation.cc:769] Sending connection context
0811 17:21:58.885674 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 17:21:58.886080 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":176,"threads_started":1}
I20260811 17:21:58.888885 30326 heartbeater.cc:347] Connected to a master server at 127.9.157.190:37477
I20260811 17:21:58.889181 30326 heartbeater.cc:464] Registering TS with master...
I20260811 17:21:58.889814 30326 heartbeater.cc:511] Master 127.9.157.190:37477 requested a full tablet report, sending...
W20260811 17:21:58.891968 30326 heartbeater.cc:644] Failed to heartbeat to 127.9.157.190:37477 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 17:21:58.892135 30326 heartbeater.cc:464] Registering TS with master...
I20260811 17:21:58.892405 30326 heartbeater.cc:511] Master 127.9.157.190:37477 requested a full tablet report, sending...
I20260811 17:21:58.893353 30326 heartbeater.cc:464] Registering TS with master...
I20260811 17:21:58.893563 30326 heartbeater.cc:511] Master 127.9.157.190:37477 requested a full tablet report, sending...
W20260811 17:21:58.894393 30326 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 17:21:59.029846 30329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:21:59.030418 30329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:21:59.030557 30329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:21:59.037174 30329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:21:59.037379 30329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:21:59.037526 30329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:21:59.046525 30329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.157.148:56467/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:44923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=32951
--tserver_master_addrs=127.9.157.190:37477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:21:59.048249 30329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:21:59.049882 30329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:21:59.064443 30337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:21:59.064443 30340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:21:59.064625 30329 server_base.cc:1034] running on GCE node
W20260811 17:21:59.067898 30336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:21:59.068631 30329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:21:59.069614 30329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:21:59.070891 30329 hybrid_clock.cc:648] HybridClock initialized: now 1786468919070812 us; error 59 us; skew 500 ppm
Aug 11 17:21:59 dist-test-slave-mjjr krb5kdc[28396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468919, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:21:59.074715 30329 init.cc:377] Logged in from keytab as kudu/127.9.157.131@KRBTEST.COM (short username kudu)
I20260811 17:21:59.076066 30329 webserver.cc:460] Webserver started at http://127.9.157.131:32951/ using document root <none> and password file <none>
I20260811 17:21:59.076920 30329 fs_manager.cc:359] Metadata directory not provided
I20260811 17:21:59.077050 30329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 17:21:59 dist-test-slave-mjjr krb5kdc[28396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468919, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for HTTP/127.9.157.148@KRBTEST.COM
I20260811 17:21:59.104254 30329 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 17:21:59.115015 30351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:21:59.117144 30329 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.003s sys 0.000s
I20260811 17:21:59.117365 30329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "ffc41ec094fc41ba81ce936bfeb92cec"
format_stamp: "Formatted at 2026-08-11 17:21:57 on dist-test-slave-mjjr"
server_key: "f8dd7fed6695bb1e95bcc8411480b5ed"
server_key_iv: "671a3b876258983e385d43caa4afab49"
server_key_version: "kuduclusterkey@0"
I20260811 17:21:59.118036 30329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:21:59.165987 30329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:21:59.170293 30329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:21:59.170563 30329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:21:59.171602 30329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:21:59.173055 30329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:21:59.173136 30329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:21:59.173213 30329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:21:59.173243 30329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:21:59.187002 30329 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:44923
I20260811 17:21:59.187312 30515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:44923 every 8 connection(s)
I20260811 17:21:59.188669 30329 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:21:59.198689 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 30329
Aug 11 17:21:59 dist-test-slave-mjjr krb5kdc[28396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468919, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:21:59.216768 30521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:59.199598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50050 (local address 127.9.157.190:37477)
0811 17:21:59.201592 (+ 1994us) server_negotiation.cc:206] Beginning negotiation
0811 17:21:59.201607 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 17:21:59.202314 (+ 707us) server_negotiation.cc:407] Connection header received
0811 17:21:59.202365 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:21:59.202370 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:21:59.202656 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:21:59.202756 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:21:59.204328 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:59.205193 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:59.206576 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:59.206958 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:59.211807 (+ 4849us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:59.212007 (+ 200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:21:59.212011 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:21:59.212013 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:21:59.212031 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:21:59.214055 (+ 2024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:21:59.214732 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:21:59.214736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:21:59.214738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:21:59.214838 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:21:59.215199 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:21:59.215203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:21:59.215205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:21:59.215471 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:21:59.215671 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 17:21:59.216440 (+ 769us) server_negotiation.cc:299] Negotiation successful
0811 17:21:59.216581 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1883,"thread_start_us":152,"threads_started":1}
I20260811 17:21:59.222097 30123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:59.193224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:36507 (local address 127.9.157.190:37477)
0811 17:21:59.193380 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 17:21:59.193387 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:21:59.193409 (+ 22us) server_negotiation.cc:407] Connection header received
0811 17:21:59.194992 (+ 1583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:21:59.194997 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:21:59.195079 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:21:59.195193 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:21:59.197088 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:59.197946 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:59.201570 (+ 3624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:59.201942 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:59.202042 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:59.216532 (+ 14490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:21:59.216537 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:21:59.216541 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:21:59.216574 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:21:59.218378 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:21:59.219186 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:21:59.219190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:21:59.219192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:21:59.219292 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:21:59.220497 (+ 1205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:21:59.220501 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:21:59.220503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:21:59.220745 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:21:59.221039 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0811 17:21:59.221764 (+ 725us) server_negotiation.cc:299] Negotiation successful
0811 17:21:59.221932 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 17:21:59.223290 30518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:59.191760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:37477 (local address 127.9.157.131:36507)
0811 17:21:59.192512 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0811 17:21:59.192585 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 17:21:59.194642 (+ 2057us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:21:59.195492 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:21:59.195505 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:21:59.195990 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:21:59.196830 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:59.196858 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:59.198177 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:59.198182 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:59.201345 (+ 3163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:59.201368 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:59.202330 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:59.202334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:59.202587 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:59.207199 (+ 4612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:21:59.207233 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:21:59.216265 (+ 9032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:21:59.218588 (+ 2323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:21:59.218596 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:21:59.218617 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:21:59.219047 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:21:59.220053 (+ 1006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:21:59.220057 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:21:59.220059 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:21:59.220321 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:21:59.221144 (+ 823us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:21:59.221153 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:21:59.221465 (+ 312us) client_negotiation.cc:769] Sending connection context
0811 17:21:59.221704 (+ 239us) client_negotiation.cc:240] Negotiation successful
0811 17:21:59.222087 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":213,"threads_started":1}
I20260811 17:21:59.224673 30516 heartbeater.cc:347] Connected to a master server at 127.9.157.190:37477
I20260811 17:21:59.224972 30516 heartbeater.cc:464] Registering TS with master...
I20260811 17:21:59.225606 30516 heartbeater.cc:511] Master 127.9.157.190:37477 requested a full tablet report, sending...
I20260811 17:21:59.227123 29826 ts_manager.cc:194] Registered new tserver with Master: ffc41ec094fc41ba81ce936bfeb92cec (127.9.157.131:44923)
I20260811 17:21:59.228475 29826 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.131@KRBTEST.COM'} at 127.9.157.131:36507
I20260811 17:21:59.578218 30121 heartbeater.cc:464] Registering TS with master...
I20260811 17:21:59.578594 30121 heartbeater.cc:511] Master 127.9.157.190:37477 requested a full tablet report, sending...
I20260811 17:21:59.579571 29826 ts_manager.cc:194] Registered new tserver with Master: c2a9d4a3347f4ebb95194e514cda12be (127.9.157.129:40103)
I20260811 17:21:59.580246 29826 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.129@KRBTEST.COM'} at 127.9.157.129:49539
Aug 11 17:21:59 dist-test-slave-mjjr krb5kdc[28396](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786468919, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:21:59.894676 30326 heartbeater.cc:464] Registering TS with master...
I20260811 17:21:59.895072 30326 heartbeater.cc:511] Master 127.9.157.190:37477 requested a full tablet report, sending...
I20260811 17:21:59.896060 29826 ts_manager.cc:194] Registered new tserver with Master: 31b3f97b137246a9bfb86e6d4a79258a (127.9.157.130:41831)
I20260811 17:21:59.896822 29826 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.130@KRBTEST.COM'} at 127.9.157.130:45147
I20260811 17:21:59.901192 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:21:59.914149 30545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:59.903002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50058 (local address 127.9.157.190:37477)
0811 17:21:59.904104 (+ 1102us) server_negotiation.cc:206] Beginning negotiation
0811 17:21:59.904108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:21:59.904121 (+ 13us) server_negotiation.cc:407] Connection header received
0811 17:21:59.904160 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:21:59.904162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:21:59.904205 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:21:59.904307 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:21:59.905070 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:59.905669 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:59.906969 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:59.907334 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:59.907446 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:59.909347 (+ 1901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:21:59.909352 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:21:59.909354 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:21:59.909392 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:21:59.911363 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:21:59.911980 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:21:59.911984 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:21:59.911987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:21:59.912086 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:21:59.912462 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:21:59.912466 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:21:59.912468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:21:59.912750 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:21:59.912906 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 17:21:59.913694 (+ 788us) server_negotiation.cc:299] Negotiation successful
0811 17:21:59.913857 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":115,"threads_started":1}
I20260811 17:21:59.920709 29826 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50058:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:21:59.926484 29826 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:21:59.947162 30564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:59.939190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:44923 (local address 127.0.0.1:57650)
0811 17:21:59.940778 (+ 1588us) negotiation.cc:107] Waiting for socket to connect
0811 17:21:59.940842 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 17:21:59.940997 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:21:59.941851 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:21:59.941858 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:21:59.941896 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:21:59.942266 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:59.942276 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:59.943675 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:59.943678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:59.945085 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:59.945095 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:59.946516 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:59.946522 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:59.946642 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:59.946688 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 17:21:59.946820 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 17:21:59.946976 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1376,"mutex_wait_us":109,"spinlock_wait_cycles":4224,"thread_start_us":153,"threads_started":1}
I20260811 17:21:59.947599 30567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:59.940257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34162 (local address 127.9.157.130:41831)
0811 17:21:59.940627 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0811 17:21:59.940637 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 17:21:59.941290 (+ 653us) server_negotiation.cc:407] Connection header received
0811 17:21:59.941407 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:21:59.941415 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:21:59.941655 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:21:59.941831 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:21:59.942454 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:59.943750 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:59.945374 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:59.946314 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:59.946476 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:59.946652 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 17:21:59.947263 (+ 611us) server_negotiation.cc:299] Negotiation successful
0811 17:21:59.947425 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":98,"threads_started":1}
I20260811 17:21:59.947863 30566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:59.940054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57650 (local address 127.9.157.131:44923)
0811 17:21:59.940839 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0811 17:21:59.940846 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:21:59.941160 (+ 314us) server_negotiation.cc:407] Connection header received
0811 17:21:59.941256 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:21:59.941262 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:21:59.941495 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:21:59.941650 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:21:59.942419 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:59.943513 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:59.945242 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:59.946134 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:59.946215 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:59.946384 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 17:21:59.947553 (+ 1169us) server_negotiation.cc:299] Negotiation successful
0811 17:21:59.947703 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":138,"threads_started":1}
I20260811 17:21:59.948678 30563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:59.939197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:40103 (local address 127.0.0.1:57184)
0811 17:21:59.940955 (+ 1758us) negotiation.cc:107] Waiting for socket to connect
0811 17:21:59.940968 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:21:59.941041 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:21:59.941851 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:21:59.941858 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:21:59.941898 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:21:59.942282 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:59.942289 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:59.944756 (+ 2467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:59.944759 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:59.945839 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:59.945850 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:59.948213 (+ 2363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:59.948217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:59.948337 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:59.948361 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 17:21:59.948445 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 17:21:59.948538 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1698,"mutex_wait_us":474,"spinlock_wait_cycles":7168,"thread_start_us":108,"threads_started":1}
I20260811 17:21:59.948761 30561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:59.939373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57184 (local address 127.9.157.129:40103)
0811 17:21:59.941312 (+ 1939us) server_negotiation.cc:206] Beginning negotiation
0811 17:21:59.941319 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:21:59.941339 (+ 20us) server_negotiation.cc:407] Connection header received
0811 17:21:59.941405 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:21:59.941409 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:21:59.941558 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:21:59.941676 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:21:59.942419 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:59.944612 (+ 2193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:59.945996 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:21:59.947836 (+ 1840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:21:59.947963 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:59.948113 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 17:21:59.948515 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 17:21:59.948623 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1818,"thread_start_us":88,"threads_started":1}
I20260811 17:21:59.948091 30556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:21:59.940073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:41831 (local address 127.0.0.1:34162)
0811 17:21:59.941017 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0811 17:21:59.941034 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:21:59.941165 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:21:59.941950 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:21:59.941952 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:21:59.941962 (+ 10us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:21:59.942269 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:59.942282 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:59.943913 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:59.943918 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:59.945182 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:21:59.945198 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:21:59.946731 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:21:59.946736 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:21:59.946848 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:21:59.946882 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 17:21:59.946968 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 17:21:59.947065 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":865,"mutex_wait_us":109,"thread_start_us":103,"threads_started":1}
I20260811 17:21:59.951867 30228 tablet_service.cc:1467] Processing CreateTablet for tablet df1577f0496a4e5ba0d298d5d49b1bcf (DEFAULT_TABLE table=test-table [id=be0c4116407d41e29fd96b08d475ea37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:21:59.951879 29998 tablet_service.cc:1467] Processing CreateTablet for tablet df1577f0496a4e5ba0d298d5d49b1bcf (DEFAULT_TABLE table=test-table [id=be0c4116407d41e29fd96b08d475ea37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:21:59.951865 30416 tablet_service.cc:1467] Processing CreateTablet for tablet df1577f0496a4e5ba0d298d5d49b1bcf (DEFAULT_TABLE table=test-table [id=be0c4116407d41e29fd96b08d475ea37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:21:59.953121 29998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df1577f0496a4e5ba0d298d5d49b1bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:21:59.953125 30416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df1577f0496a4e5ba0d298d5d49b1bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:21:59.953125 30228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df1577f0496a4e5ba0d298d5d49b1bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:21:59.963503 30571 tablet_bootstrap.cc:492] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec: Bootstrap starting.
I20260811 17:21:59.964007 30572 tablet_bootstrap.cc:492] T df1577f0496a4e5ba0d298d5d49b1bcf P 31b3f97b137246a9bfb86e6d4a79258a: Bootstrap starting.
I20260811 17:21:59.966521 30571 tablet_bootstrap.cc:654] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec: Neither blocks nor log segments found. Creating new log.
I20260811 17:21:59.967492 30572 tablet_bootstrap.cc:654] T df1577f0496a4e5ba0d298d5d49b1bcf P 31b3f97b137246a9bfb86e6d4a79258a: Neither blocks nor log segments found. Creating new log.
I20260811 17:21:59.967684 30571 log.cc:824] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec: Log is configured to *not* fsync() on all Append() calls
I20260811 17:21:59.968633 30572 log.cc:824] T df1577f0496a4e5ba0d298d5d49b1bcf P 31b3f97b137246a9bfb86e6d4a79258a: Log is configured to *not* fsync() on all Append() calls
I20260811 17:21:59.970805 30571 tablet_bootstrap.cc:492] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec: No bootstrap required, opened a new log
I20260811 17:21:59.971097 30571 ts_tablet_manager.cc:1397] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 17:21:59.971493 30572 tablet_bootstrap.cc:492] T df1577f0496a4e5ba0d298d5d49b1bcf P 31b3f97b137246a9bfb86e6d4a79258a: No bootstrap required, opened a new log
I20260811 17:21:59.971740 30572 ts_tablet_manager.cc:1397] T df1577f0496a4e5ba0d298d5d49b1bcf P 31b3f97b137246a9bfb86e6d4a79258a: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 17:21:59.976895 30571 raft_consensus.cc:348] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffc41ec094fc41ba81ce936bfeb92cec" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44923 } } peers { permanent_uuid: "c2a9d4a3347f4ebb95194e514cda12be" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40103 } } peers { permanent_uuid: "31b3f97b137246a9bfb86e6d4a79258a" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 41831 } }
I20260811 17:21:59.977455 30571 raft_consensus.cc:374] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:21:59.977584 30571 raft_consensus.cc:729] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffc41ec094fc41ba81ce936bfeb92cec, State: Initialized, Role: FOLLOWER
I20260811 17:21:59.977795 30572 raft_consensus.cc:348] T df1577f0496a4e5ba0d298d5d49b1bcf P 31b3f97b137246a9bfb86e6d4a79258a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffc41ec094fc41ba81ce936bfeb92cec" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44923 } } peers { permanent_uuid: "c2a9d4a3347f4ebb95194e514cda12be" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40103 } } peers { permanent_uuid: "31b3f97b137246a9bfb86e6d4a79258a" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 41831 } }
I20260811 17:21:59.978082 30572 raft_consensus.cc:374] T df1577f0496a4e5ba0d298d5d49b1bcf P 31b3f97b137246a9bfb86e6d4a79258a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:21:59.978150 30572 raft_consensus.cc:729] T df1577f0496a4e5ba0d298d5d49b1bcf P 31b3f97b137246a9bfb86e6d4a79258a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31b3f97b137246a9bfb86e6d4a79258a, State: Initialized, Role: FOLLOWER
I20260811 17:21:59.978431 30571 consensus_queue.cc:260] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffc41ec094fc41ba81ce936bfeb92cec" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44923 } } peers { permanent_uuid: "c2a9d4a3347f4ebb95194e514cda12be" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40103 } } peers { permanent_uuid: "31b3f97b137246a9bfb86e6d4a79258a" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 41831 } }
I20260811 17:21:59.978621 30572 consensus_queue.cc:260] T df1577f0496a4e5ba0d298d5d49b1bcf P 31b3f97b137246a9bfb86e6d4a79258a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffc41ec094fc41ba81ce936bfeb92cec" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44923 } } peers { permanent_uuid: "c2a9d4a3347f4ebb95194e514cda12be" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40103 } } peers { permanent_uuid: "31b3f97b137246a9bfb86e6d4a79258a" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 41831 } }
I20260811 17:21:59.979887 30572 ts_tablet_manager.cc:1428] T df1577f0496a4e5ba0d298d5d49b1bcf P 31b3f97b137246a9bfb86e6d4a79258a: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260811 17:21:59.979887 30571 ts_tablet_manager.cc:1428] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260811 17:21:59.980347 30516 heartbeater.cc:503] Master 127.9.157.190:37477 was elected leader, sending a full tablet report...
I20260811 17:21:59.980347 30326 heartbeater.cc:503] Master 127.9.157.190:37477 was elected leader, sending a full tablet report...
I20260811 17:21:59.982333 30575 tablet_bootstrap.cc:492] T df1577f0496a4e5ba0d298d5d49b1bcf P c2a9d4a3347f4ebb95194e514cda12be: Bootstrap starting.
I20260811 17:21:59.985249 30575 tablet_bootstrap.cc:654] T df1577f0496a4e5ba0d298d5d49b1bcf P c2a9d4a3347f4ebb95194e514cda12be: Neither blocks nor log segments found. Creating new log.
I20260811 17:21:59.986361 30575 log.cc:824] T df1577f0496a4e5ba0d298d5d49b1bcf P c2a9d4a3347f4ebb95194e514cda12be: Log is configured to *not* fsync() on all Append() calls
I20260811 17:21:59.989264 30575 tablet_bootstrap.cc:492] T df1577f0496a4e5ba0d298d5d49b1bcf P c2a9d4a3347f4ebb95194e514cda12be: No bootstrap required, opened a new log
I20260811 17:21:59.989540 30575 ts_tablet_manager.cc:1397] T df1577f0496a4e5ba0d298d5d49b1bcf P c2a9d4a3347f4ebb95194e514cda12be: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.001s
I20260811 17:21:59.994707 30575 raft_consensus.cc:348] T df1577f0496a4e5ba0d298d5d49b1bcf P c2a9d4a3347f4ebb95194e514cda12be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffc41ec094fc41ba81ce936bfeb92cec" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44923 } } peers { permanent_uuid: "c2a9d4a3347f4ebb95194e514cda12be" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40103 } } peers { permanent_uuid: "31b3f97b137246a9bfb86e6d4a79258a" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 41831 } }
I20260811 17:21:59.995252 30575 raft_consensus.cc:374] T df1577f0496a4e5ba0d298d5d49b1bcf P c2a9d4a3347f4ebb95194e514cda12be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:21:59.995473 30575 raft_consensus.cc:729] T df1577f0496a4e5ba0d298d5d49b1bcf P c2a9d4a3347f4ebb95194e514cda12be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2a9d4a3347f4ebb95194e514cda12be, State: Initialized, Role: FOLLOWER
I20260811 17:21:59.996251 30575 consensus_queue.cc:260] T df1577f0496a4e5ba0d298d5d49b1bcf P c2a9d4a3347f4ebb95194e514cda12be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffc41ec094fc41ba81ce936bfeb92cec" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44923 } } peers { permanent_uuid: "c2a9d4a3347f4ebb95194e514cda12be" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40103 } } peers { permanent_uuid: "31b3f97b137246a9bfb86e6d4a79258a" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 41831 } }
I20260811 17:21:59.997202 30121 heartbeater.cc:503] Master 127.9.157.190:37477 was elected leader, sending a full tablet report...
I20260811 17:21:59.997666 30575 ts_tablet_manager.cc:1428] T df1577f0496a4e5ba0d298d5d49b1bcf P c2a9d4a3347f4ebb95194e514cda12be: Time spent starting tablet: real 0.008s user 0.005s sys 0.001s
W20260811 17:22:00.044945 30122 tablet.cc:2369] T df1577f0496a4e5ba0d298d5d49b1bcf P c2a9d4a3347f4ebb95194e514cda12be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:22:00.088841 30576 raft_consensus.cc:482] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:22:00.089097 30576 raft_consensus.cc:504] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffc41ec094fc41ba81ce936bfeb92cec" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44923 } } peers { permanent_uuid: "c2a9d4a3347f4ebb95194e514cda12be" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40103 } } peers { permanent_uuid: "31b3f97b137246a9bfb86e6d4a79258a" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 41831 } }
I20260811 17:22:00.090533 30576 leader_election.cc:288] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2a9d4a3347f4ebb95194e514cda12be (127.9.157.129:40103), 31b3f97b137246a9bfb86e6d4a79258a (127.9.157.130:41831)
I20260811 17:22:00.095974 30561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:00.090993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:47809 (local address 127.9.157.129:40103)
0811 17:22:00.091126 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:00.091131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:00.091401 (+ 270us) server_negotiation.cc:407] Connection header received
0811 17:22:00.091604 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:00.091608 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:00.091693 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:00.091760 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:22:00.092222 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:00.092981 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:00.094556 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:00.095147 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:00.095309 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:00.095365 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:00.095745 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 17:22:00.095832 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 17:22:00.097857 30567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:00.091782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:47221 (local address 127.9.157.130:41831)
0811 17:22:00.092030 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:00.092035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:00.093068 (+ 1033us) server_negotiation.cc:407] Connection header received
0811 17:22:00.093294 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:00.093298 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:00.093353 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:00.093410 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:22:00.094039 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:00.094811 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:00.096425 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:00.097023 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:00.097130 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:00.097191 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:00.097598 (+ 407us) server_negotiation.cc:299] Negotiation successful
0811 17:22:00.097691 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
I20260811 17:22:00.098423 30584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:00.091329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:41831 (local address 127.9.157.131:47221)
0811 17:22:00.092965 (+ 1636us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:00.092982 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:00.093147 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:00.093756 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:00.093759 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:00.093777 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:22:00.093901 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:00.093907 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:00.094964 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:00.094967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:00.096238 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:00.096263 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:00.097315 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:00.097318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:00.097437 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:00.097465 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 17:22:00.098105 (+ 640us) client_negotiation.cc:240] Negotiation successful
0811 17:22:00.098233 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1561,"thread_start_us":94,"threads_started":1}
I20260811 17:22:00.099254 30235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df1577f0496a4e5ba0d298d5d49b1bcf" candidate_uuid: "ffc41ec094fc41ba81ce936bfeb92cec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31b3f97b137246a9bfb86e6d4a79258a" is_pre_election: true
I20260811 17:22:00.099676 30235 raft_consensus.cc:2454] T df1577f0496a4e5ba0d298d5d49b1bcf P 31b3f97b137246a9bfb86e6d4a79258a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffc41ec094fc41ba81ce936bfeb92cec in term 0.
I20260811 17:22:00.100322 30355 leader_election.cc:302] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31b3f97b137246a9bfb86e6d4a79258a, ffc41ec094fc41ba81ce936bfeb92cec; no voters:
I20260811 17:22:00.100695 30576 raft_consensus.cc:2781] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:22:00.100773 30576 raft_consensus.cc:482] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:22:00.100824 30576 raft_consensus.cc:3037] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:22:00.102334 30576 raft_consensus.cc:504] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffc41ec094fc41ba81ce936bfeb92cec" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44923 } } peers { permanent_uuid: "c2a9d4a3347f4ebb95194e514cda12be" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40103 } } peers { permanent_uuid: "31b3f97b137246a9bfb86e6d4a79258a" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 41831 } }
I20260811 17:22:00.102847 30576 leader_election.cc:288] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec [CANDIDATE]: Term 1 election: Requested vote from peers c2a9d4a3347f4ebb95194e514cda12be (127.9.157.129:40103), 31b3f97b137246a9bfb86e6d4a79258a (127.9.157.130:41831)
I20260811 17:22:00.103686 30235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df1577f0496a4e5ba0d298d5d49b1bcf" candidate_uuid: "ffc41ec094fc41ba81ce936bfeb92cec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31b3f97b137246a9bfb86e6d4a79258a"
I20260811 17:22:00.103856 30235 raft_consensus.cc:3037] T df1577f0496a4e5ba0d298d5d49b1bcf P 31b3f97b137246a9bfb86e6d4a79258a [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:22:00.104113 30583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:00.090844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:40103 (local address 127.9.157.131:47809)
0811 17:22:00.091303 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:00.091318 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:00.091485 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:00.091914 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:00.091917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:00.091937 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:22:00.092094 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:00.092102 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:00.093132 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:00.093135 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:00.094412 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:00.094424 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:00.095469 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:00.095472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:00.095572 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:00.095598 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 17:22:00.099384 (+ 3786us) client_negotiation.cc:240] Negotiation successful
0811 17:22:00.103942 (+ 4558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":137,"threads_started":1}
I20260811 17:22:00.105346 30235 raft_consensus.cc:2454] T df1577f0496a4e5ba0d298d5d49b1bcf P 31b3f97b137246a9bfb86e6d4a79258a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffc41ec094fc41ba81ce936bfeb92cec in term 1.
I20260811 17:22:00.106047 30355 leader_election.cc:302] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31b3f97b137246a9bfb86e6d4a79258a, ffc41ec094fc41ba81ce936bfeb92cec; no voters:
I20260811 17:22:00.106412 30576 raft_consensus.cc:2781] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:22:00.106698 30576 raft_consensus.cc:686] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec [term 1 LEADER]: Becoming Leader. State: Replica: ffc41ec094fc41ba81ce936bfeb92cec, State: Running, Role: LEADER
I20260811 17:22:00.107143 30576 consensus_queue.cc:237] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffc41ec094fc41ba81ce936bfeb92cec" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44923 } } peers { permanent_uuid: "c2a9d4a3347f4ebb95194e514cda12be" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40103 } } peers { permanent_uuid: "31b3f97b137246a9bfb86e6d4a79258a" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 41831 } }
I20260811 17:22:00.107380 30046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df1577f0496a4e5ba0d298d5d49b1bcf" candidate_uuid: "ffc41ec094fc41ba81ce936bfeb92cec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2a9d4a3347f4ebb95194e514cda12be" is_pre_election: true
I20260811 17:22:00.107735 30046 raft_consensus.cc:2454] T df1577f0496a4e5ba0d298d5d49b1bcf P c2a9d4a3347f4ebb95194e514cda12be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffc41ec094fc41ba81ce936bfeb92cec in term 0.
I20260811 17:22:00.108119 30044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df1577f0496a4e5ba0d298d5d49b1bcf" candidate_uuid: "ffc41ec094fc41ba81ce936bfeb92cec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2a9d4a3347f4ebb95194e514cda12be"
I20260811 17:22:00.108253 30044 raft_consensus.cc:3037] T df1577f0496a4e5ba0d298d5d49b1bcf P c2a9d4a3347f4ebb95194e514cda12be [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:22:00.109706 30044 raft_consensus.cc:2454] T df1577f0496a4e5ba0d298d5d49b1bcf P c2a9d4a3347f4ebb95194e514cda12be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffc41ec094fc41ba81ce936bfeb92cec in term 1.
I20260811 17:22:00.111229 29824 catalog_manager.cc:5495] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec reported cstate change: term changed from 0 to 1, leader changed from <none> to ffc41ec094fc41ba81ce936bfeb92cec (127.9.157.131). New cstate: current_term: 1 leader_uuid: "ffc41ec094fc41ba81ce936bfeb92cec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffc41ec094fc41ba81ce936bfeb92cec" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2a9d4a3347f4ebb95194e514cda12be" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31b3f97b137246a9bfb86e6d4a79258a" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 41831 } health_report { overall_health: UNKNOWN } } }
W20260811 17:22:00.114831 30327 tablet.cc:2369] T df1577f0496a4e5ba0d298d5d49b1bcf P 31b3f97b137246a9bfb86e6d4a79258a: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 17:22:00 dist-test-slave-mjjr krb5kdc[28396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468919, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for HTTP/127.9.157.148@KRBTEST.COM
I20260811 17:22:00.166308 30566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:00.161150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57660 (local address 127.9.157.131:44923)
0811 17:22:00.161299 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:00.161304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:00.161320 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:22:00.161430 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:00.161434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:00.161508 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:00.161625 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:22:00.162057 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:00.162832 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:00.164082 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:00.164471 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:00.164594 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:00.165102 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:22:00.165760 (+ 658us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:22:00.165902 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:00.166070 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 17:22:00.166145 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 17:22:00.174607 30235 raft_consensus.cc:1261] T df1577f0496a4e5ba0d298d5d49b1bcf P 31b3f97b137246a9bfb86e6d4a79258a [term 1 FOLLOWER]: Refusing update from remote peer ffc41ec094fc41ba81ce936bfeb92cec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:22:00.176366 30044 raft_consensus.cc:1261] T df1577f0496a4e5ba0d298d5d49b1bcf P c2a9d4a3347f4ebb95194e514cda12be [term 1 FOLLOWER]: Refusing update from remote peer ffc41ec094fc41ba81ce936bfeb92cec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:22:00.177770 30586 consensus_queue.cc:1035] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec [LEADER]: Connected to new peer: Peer: permanent_uuid: "31b3f97b137246a9bfb86e6d4a79258a" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 41831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:22:00.178412 30586 consensus_queue.cc:1035] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2a9d4a3347f4ebb95194e514cda12be" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:22:00.195447 30591 mvcc.cc:205] Tried to move back new op lower bound from 7317376697025327104 to 7317376696764112896. Current Snapshot: MvccSnapshot[applied={T|T < 7317376697025327104}]
I20260811 17:22:00.197389 30592 mvcc.cc:205] Tried to move back new op lower bound from 7317376697025327104 to 7317376696764112896. Current Snapshot: MvccSnapshot[applied={T|T < 7317376697025327104}]
I20260811 17:22:00.209474 30596 mvcc.cc:205] Tried to move back new op lower bound from 7317376697025327104 to 7317376696764112896. Current Snapshot: MvccSnapshot[applied={T|T < 7317376697025327104}]
I20260811 17:22:00.217149 29825 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:22:00.217338 29825 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:22:00.220840 29825 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce461148e94f403dac04a883809f14ba: Sending DeleteTablet for 3 replicas of tablet df1577f0496a4e5ba0d298d5d49b1bcf
I20260811 17:22:00.222137 30416 tablet_service.cc:1514] Processing DeleteTablet for tablet df1577f0496a4e5ba0d298d5d49b1bcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:22:00 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:57650
I20260811 17:22:00.223418 29998 tablet_service.cc:1514] Processing DeleteTablet for tablet df1577f0496a4e5ba0d298d5d49b1bcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:22:00 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:57184
I20260811 17:22:00.223996 30228 tablet_service.cc:1514] Processing DeleteTablet for tablet df1577f0496a4e5ba0d298d5d49b1bcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:22:00 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:34162
I20260811 17:22:00.225260 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 29881
I20260811 17:22:00.226481 30606 tablet_replica.cc:330] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec: stopping tablet replica
I20260811 17:22:00.226897 30606 raft_consensus.cc:2229] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:22:00.236004 30609 tablet_replica.cc:330] T df1577f0496a4e5ba0d298d5d49b1bcf P 31b3f97b137246a9bfb86e6d4a79258a: stopping tablet replica
I20260811 17:22:00.236491 30609 raft_consensus.cc:2229] T df1577f0496a4e5ba0d298d5d49b1bcf P 31b3f97b137246a9bfb86e6d4a79258a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:22:00.236871 30609 raft_consensus.cc:2258] T df1577f0496a4e5ba0d298d5d49b1bcf P 31b3f97b137246a9bfb86e6d4a79258a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:22:00.239985 30606 raft_consensus.cc:2258] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:22:00.241931 30606 ts_tablet_manager.cc:1905] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:22:00.244325 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 30126
W20260811 17:22:00.244361 29811 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.157.129:40103 (error 108)
I20260811 17:22:00.251111 30606 ts_tablet_manager.cc:1918] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:22:00.251382 30606 log.cc:1196] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786468743878666-9846-0/minicluster-data/ts-2/wal/wals/df1577f0496a4e5ba0d298d5d49b1bcf
I20260811 17:22:00.251864 30606 ts_tablet_manager.cc:1939] T df1577f0496a4e5ba0d298d5d49b1bcf P ffc41ec094fc41ba81ce936bfeb92cec: Deleting consensus metadata
I20260811 17:22:00.253501 29807 catalog_manager.cc:4841] TS ffc41ec094fc41ba81ce936bfeb92cec (127.9.157.131:44923): tablet df1577f0496a4e5ba0d298d5d49b1bcf (table test-table [id=be0c4116407d41e29fd96b08d475ea37]) successfully deleted
I20260811 17:22:00.257388 30556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:00.257017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:40103 (local address 127.0.0.1:57186)
0811 17:22:00.257216 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:00.257298 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.129:40103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260811 17:22:00.257717 29811 catalog_manager.cc:4584] TS c2a9d4a3347f4ebb95194e514cda12be (127.9.157.129:40103): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet df1577f0496a4e5ba0d298d5d49b1bcf: Network error: Client connection negotiation failed: client connection to 127.9.157.129:40103: connect: Connection refused (error 111)
W20260811 17:22:00.270377 30556 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:22:00.269077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:41831 (local address 127.0.0.1:34174)
0811 17:22:00.269318 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:00.269337 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:00.269455 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:00.270225 (+ 770us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.130:41831: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":147}
I20260811 17:22:00.273082 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 30329
I20260811 17:22:00.288640 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 29784
I20260811 17:22:00.304922 9846 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 17:22:00.651376 9846 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 17:22:00.651532 9846 mini_ranger.cc:66] Stopping Ranger...
I20260811 17:22:01.008060 9846 mini_ranger.cc:68] Stopped Ranger
2026-08-11 17:22:01.008 UTC [28437] LOG: received smart shutdown request
2026-08-11 17:22:01.013 UTC [28437] LOG: background worker "logical replication launcher" (PID 28455) exited with exit code 1
2026-08-11 17:22:01.013 UTC [28446] LOG: shutting down
2026-08-11 17:22:01.196 UTC [28437] LOG: database system is shut down
2026-08-11T17:22:01Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (53761 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:22:01 dist-test-slave-mjjr krb5kdc[30622](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:22:01 dist-test-slave-mjjr krb5kdc[30622](info): set up 2 sockets
Aug 11 17:22:01 dist-test-slave-mjjr krb5kdc[30622](info): commencing operation
krb5kdc: starting...
W20260811 17:22:02.602866 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:22:02 dist-test-slave-mjjr krb5kdc[30622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:22:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:22:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:22:02.764055 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:34917
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:42309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:34917,127.9.157.189:43479,127.9.157.188:32769
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:22:02.893834 30644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:22:02.894183 30644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:22:02.894229 30644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:22:02.898568 30644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:22:02.898662 30644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:22:02.898684 30644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:22:02.898701 30644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:22:02.898717 30644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:22:02.904906 30644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:42309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:34917,127.9.157.189:43479,127.9.157.188:32769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:34917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:22:02.906224 30644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:22:02.907467 30644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:22:02.917879 30652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:02.917913 30656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:02.917883 30653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:02.918236 30644 server_base.cc:1034] running on GCE node
I20260811 17:22:02.919113 30644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:22:02.920558 30644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:22:02.921793 30644 hybrid_clock.cc:648] HybridClock initialized: now 1786468922921768 us; error 46 us; skew 500 ppm
Aug 11 17:22:02 dist-test-slave-mjjr krb5kdc[30622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468922, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:22:02.925220 30644 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:22:02.926295 30644 webserver.cc:460] Webserver started at http://127.9.157.190:43827/ using document root <none> and password file <none>
I20260811 17:22:02.926908 30644 fs_manager.cc:359] Metadata directory not provided
I20260811 17:22:02.926976 30644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:22:02.927165 30644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:22:02.929538 30644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "d1aac9a869bf4a4fb6689722529b9a73"
format_stamp: "Formatted at 2026-08-11 17:22:02 on dist-test-slave-mjjr"
server_key: "9ddb1157b1e89f62c526294ee2462bfd"
server_key_iv: "fb1a1caac21fcf2d1eafd7cd195335c3"
server_key_version: "encryptionkey@0"
I20260811 17:22:02.930171 30644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "d1aac9a869bf4a4fb6689722529b9a73"
format_stamp: "Formatted at 2026-08-11 17:22:02 on dist-test-slave-mjjr"
server_key: "9ddb1157b1e89f62c526294ee2462bfd"
server_key_iv: "fb1a1caac21fcf2d1eafd7cd195335c3"
server_key_version: "encryptionkey@0"
I20260811 17:22:02.935112 30644 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 17:22:02.938946 30665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:02.940582 30644 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:22:02.940732 30644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "d1aac9a869bf4a4fb6689722529b9a73"
format_stamp: "Formatted at 2026-08-11 17:22:02 on dist-test-slave-mjjr"
server_key: "9ddb1157b1e89f62c526294ee2462bfd"
server_key_iv: "fb1a1caac21fcf2d1eafd7cd195335c3"
server_key_version: "encryptionkey@0"
I20260811 17:22:02.940864 30644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:22:02.955140 30644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:22:02.959272 30644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:22:02.959465 30644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:22:02.967821 30644 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:34917
I20260811 17:22:02.967837 30741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:34917 every 8 connection(s)
I20260811 17:22:02.968997 30644 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:22:02.972219 30742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:22:02.973628 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 30644
I20260811 17:22:02.973784 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:22:02.974128 9846 external_mini_cluster.cc:1428] Setting key b7f13b7d9bc2b548ef0c0364c86c01d7
I20260811 17:22:02.978089 30742 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34917 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.157.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.189@KRBTEST.COM" created.
Aug 11 17:22:02 dist-test-slave-mjjr krb5kdc[30622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468922, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:22:02.997308 30745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:02.980962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37896 (local address 127.9.157.190:34917)
0811 17:22:02.981595 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:02.981617 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:02.982037 (+ 420us) server_negotiation.cc:407] Connection header received
0811 17:22:02.983255 (+ 1218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:02.983262 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:02.983550 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:02.983815 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:02.984747 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:02.985724 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:02.987067 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:02.987571 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:02.987881 (+ 310us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:02.991233 (+ 3352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:02.991241 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:02.991254 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:02.991278 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:02.993754 (+ 2476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:02.994302 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:02.994313 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:02.994314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:02.994427 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:02.994748 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:02.994753 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:02.994755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:02.995205 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:02.995354 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:02.995794 (+ 440us) server_negotiation.cc:299] Negotiation successful
0811 17:22:02.996151 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":137,"server-negotiator.queue_time_us":431,"thread_start_us":183,"threads_started":1}
I20260811 17:22:02.997305 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:02.980962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34917 (local address 127.0.0.1:37896)
0811 17:22:02.981816 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:02.981873 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:02.982850 (+ 977us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:02.983838 (+ 988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:02.983844 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:02.984241 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:22:02.984525 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:02.984540 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:02.985903 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:02.985906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:02.986902 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:02.986920 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:02.987796 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:02.987800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:02.987996 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:02.988679 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:22:02.988699 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:22:02.991038 (+ 2339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:22:02.993891 (+ 2853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:02.993903 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:02.993917 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:02.994176 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:02.994529 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:02.994532 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:02.994534 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:02.994654 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:02.995329 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:22:02.995336 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:22:02.995495 (+ 159us) client_negotiation.cc:769] Sending connection context
0811 17:22:02.995723 (+ 228us) client_negotiation.cc:240] Negotiation successful
0811 17:22:02.996099 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"mutex_wait_us":63,"spinlock_wait_cycles":65920,"thread_start_us":183,"threads_started":1}
I20260811 17:22:03.001433 30742 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.189" port: 43479 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:22:03.002667 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.002328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:43479 (local address 127.0.0.1:56398)
0811 17:22:03.002464 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.002574 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.189:43479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 17:22:03.002993 30667 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.157.189:43479: connect: Connection refused (error 111)
I20260811 17:22:03.004242 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.003963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:43479 (local address 127.0.0.1:56412)
0811 17:22:03.004086 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.004146 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.189:43479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 17:22:03.004521 30742 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.189:43479: Network error: Client connection negotiation failed: client connection to 127.9.157.189:43479: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.189.keytab.
Entry for principal kudu/127.9.157.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.189.keytab.
I20260811 17:22:03.007632 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.189:43479
--webserver_interface=127.9.157.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.189.keytab
--principal=kudu/127.9.157.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:42309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:34917,127.9.157.189:43479,127.9.157.188:32769
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:22:03.053905 30742 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.189" port: 43479 } attempt: 1
I20260811 17:22:03.055030 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.054671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:43479 (local address 127.0.0.1:56422)
0811 17:22:03.054861 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.054946 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.189:43479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 17:22:03.055894 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.055691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:43479 (local address 127.0.0.1:56428)
0811 17:22:03.055787 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.055835 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.189:43479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 17:22:03.056049 30742 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.189:43479: Network error: Client connection negotiation failed: client connection to 127.9.157.189:43479: connect: Connection refused (error 111)
I20260811 17:22:03.124222 30742 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.189" port: 43479 } attempt: 2
I20260811 17:22:03.125329 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.124929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:43479 (local address 127.0.0.1:56444)
0811 17:22:03.125178 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.125246 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.189:43479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
I20260811 17:22:03.126071 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.125852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:43479 (local address 127.0.0.1:56452)
0811 17:22:03.125963 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.126015 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.189:43479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 17:22:03.126204 30742 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.189:43479: Network error: Client connection negotiation failed: client connection to 127.9.157.189:43479: connect: Connection refused (error 111)
W20260811 17:22:03.134857 30751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:22:03.135293 30751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:22:03.135391 30751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:22:03.139904 30751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:22:03.140021 30751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:22:03.140059 30751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:22:03.140097 30751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:22:03.140149 30751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:22:03.146435 30751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:42309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:34917,127.9.157.189:43479,127.9.157.188:32769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.189.keytab
--principal=kudu/127.9.157.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.189:43479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.157.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:22:03.147894 30751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:22:03.149052 30751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:22:03.158895 30760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:03.158880 30764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:03.158887 30758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:03.159292 30751 server_base.cc:1034] running on GCE node
I20260811 17:22:03.160086 30751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:22:03.161427 30751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:22:03.162627 30751 hybrid_clock.cc:648] HybridClock initialized: now 1786468923162591 us; error 54 us; skew 500 ppm
Aug 11 17:22:03 dist-test-slave-mjjr krb5kdc[30622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468923, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:22:03.167047 30751 init.cc:377] Logged in from keytab as kudu/127.9.157.189@KRBTEST.COM (short username kudu)
I20260811 17:22:03.168483 30751 webserver.cc:460] Webserver started at http://127.9.157.189:40159/ using document root <none> and password file <none>
I20260811 17:22:03.169243 30751 fs_manager.cc:359] Metadata directory not provided
I20260811 17:22:03.169360 30751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:22:03.169631 30751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:22:03.172134 30751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-1/data/instance:
uuid: "61d8814ce09e49e08c225c7b6e88f33d"
format_stamp: "Formatted at 2026-08-11 17:22:03 on dist-test-slave-mjjr"
server_key: "fef781b70690e7b1bebd3d0b51506e55"
server_key_iv: "b9b0ecf6fd91b4a66d8a726c8cd4bf1d"
server_key_version: "encryptionkey@0"
I20260811 17:22:03.172837 30751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-1/wal/instance:
uuid: "61d8814ce09e49e08c225c7b6e88f33d"
format_stamp: "Formatted at 2026-08-11 17:22:03 on dist-test-slave-mjjr"
server_key: "fef781b70690e7b1bebd3d0b51506e55"
server_key_iv: "b9b0ecf6fd91b4a66d8a726c8cd4bf1d"
server_key_version: "encryptionkey@0"
I20260811 17:22:03.177551 30751 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:22:03.180670 30774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:03.182356 30751 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 17:22:03.182577 30751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-1/wal
uuid: "61d8814ce09e49e08c225c7b6e88f33d"
format_stamp: "Formatted at 2026-08-11 17:22:03 on dist-test-slave-mjjr"
server_key: "fef781b70690e7b1bebd3d0b51506e55"
server_key_iv: "b9b0ecf6fd91b4a66d8a726c8cd4bf1d"
server_key_version: "encryptionkey@0"
I20260811 17:22:03.182730 30751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:22:03.195571 30751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:22:03.199716 30751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:22:03.199985 30751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:22:03.208418 30751 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.189:43479
I20260811 17:22:03.208448 30857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.189:43479 every 8 connection(s)
I20260811 17:22:03.209689 30751 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-1/data/info.pb
I20260811 17:22:03.214000 30858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:22:03.217208 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 30751
I20260811 17:22:03.217438 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-1/wal/instance
I20260811 17:22:03.217492 30742 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.189" port: 43479 } attempt: 3
I20260811 17:22:03.217763 9846 external_mini_cluster.cc:1428] Setting key d4ddab9d2cbacd9b949717217b7a447f
I20260811 17:22:03.220487 30858 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34917 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 17:22:03 dist-test-slave-mjjr krb5kdc[30622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468922, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for kudu/127.9.157.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.157.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.188@KRBTEST.COM" created.
Aug 11 17:22:03 dist-test-slave-mjjr krb5kdc[30622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468923, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.189@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:22:03.235688 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.218177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:43479 (local address 127.0.0.1:56456)
0811 17:22:03.218430 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.218448 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:03.218550 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:03.221800 (+ 3250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:03.221805 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:03.222043 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:22:03.222214 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:03.222224 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.224783 (+ 2559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.224791 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:03.225849 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:03.225866 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.227388 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.227392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:03.227506 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:03.228143 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:22:03.228157 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:22:03.229540 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:22:03.232403 (+ 2863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:03.232407 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:03.232408 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:03.232664 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:03.233220 (+ 556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:03.233224 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:03.233226 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:03.233352 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:03.234660 (+ 1308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:22:03.234666 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:22:03.234807 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 17:22:03.235210 (+ 403us) client_negotiation.cc:240] Negotiation successful
0811 17:22:03.235428 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":4}
I20260811 17:22:03.237445 30860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.219066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56456 (local address 127.9.157.189:43479)
0811 17:22:03.219561 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:03.219596 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:03.219687 (+ 91us) server_negotiation.cc:407] Connection header received
0811 17:22:03.220800 (+ 1113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:03.220834 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:03.221386 (+ 552us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:03.221822 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:03.222400 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.224624 (+ 2224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.226132 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.227203 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.227396 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:03.229754 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:03.229764 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:03.229783 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:03.229842 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:03.232217 (+ 2375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:03.232894 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:03.232901 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:03.232907 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:03.233055 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:03.233471 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:03.233478 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:03.233483 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:03.234182 (+ 699us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:03.234349 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:03.235609 (+ 1260us) server_negotiation.cc:299] Negotiation successful
0811 17:22:03.236061 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":331,"spinlock_wait_cycles":10112,"thread_start_us":167,"threads_started":1}
I20260811 17:22:03.238246 30745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.222966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37912 (local address 127.9.157.190:34917)
0811 17:22:03.223100 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:03.223104 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:03.223604 (+ 500us) server_negotiation.cc:407] Connection header received
0811 17:22:03.223899 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:03.223905 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:03.223973 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:03.224068 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:03.224966 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.225520 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.227341 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.228580 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.228719 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:03.233838 (+ 5119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:03.233842 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:03.233845 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:03.233873 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:03.235901 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:03.236536 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:03.236541 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:03.236543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:03.236632 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:03.236936 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:03.236942 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:03.236946 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:03.237211 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:03.237330 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:03.237840 (+ 510us) server_negotiation.cc:299] Negotiation successful
0811 17:22:03.238003 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":47}
I20260811 17:22:03.239041 30863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.222932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34917 (local address 127.0.0.1:37912)
0811 17:22:03.223465 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.223521 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:03.223737 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:03.224144 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:03.224151 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:03.224657 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:22:03.224843 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:03.224849 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.225652 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.225655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:03.227153 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:03.227170 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.228971 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.228980 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:03.229156 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:03.229857 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:22:03.229874 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:22:03.233684 (+ 3810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:22:03.236070 (+ 2386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:03.236082 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:03.236094 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:03.236423 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:03.236729 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:03.236733 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:03.236735 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:03.236836 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:03.237393 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:22:03.237402 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:22:03.237562 (+ 160us) client_negotiation.cc:769] Sending connection context
0811 17:22:03.238475 (+ 913us) client_negotiation.cc:240] Negotiation successful
0811 17:22:03.238727 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":47,"spinlock_wait_cycles":3840,"thread_start_us":199,"threads_started":1}
I20260811 17:22:03.239888 30742 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:22:03.240424 30858 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.189" port: 43479 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:22:03.240736 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.240433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:32769 (local address 127.0.0.1:41122)
0811 17:22:03.240585 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.240653 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:32769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 17:22:03.241511 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.241250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:32769 (local address 127.0.0.1:41132)
0811 17:22:03.241387 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.241442 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:32769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 17:22:03.241663 30742 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.188:32769: Network error: Client connection negotiation failed: client connection to 127.9.157.188:32769: connect: Connection refused (error 111)
Aug 11 17:22:03 dist-test-slave-mjjr krb5kdc[30622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468923, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.189@KRBTEST.COM for kudu/127.9.157.189@KRBTEST.COM
I20260811 17:22:03.252123 30863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.240981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:43479 (local address 127.0.0.1:56470)
0811 17:22:03.241245 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.241272 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:03.241378 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:03.241822 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:03.241828 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:03.242039 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:22:03.242195 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:03.242207 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.242959 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.242964 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:03.243971 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:03.243987 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.244558 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.244562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:03.244721 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:03.245445 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:22:03.245458 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:22:03.248059 (+ 2601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:22:03.250203 (+ 2144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:03.250208 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:03.250210 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:03.250508 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:03.250914 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:03.250917 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:03.250918 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:03.251003 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:03.251457 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:22:03.251460 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:22:03.251555 (+ 95us) client_negotiation.cc:769] Sending connection context
0811 17:22:03.251708 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 17:22:03.251927 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":1}
I20260811 17:22:03.252635 30860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.241210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56470 (local address 127.9.157.189:43479)
0811 17:22:03.241445 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:03.241450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:03.241464 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:22:03.241503 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:03.241506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:03.241559 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:03.241796 (+ 237us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:03.242338 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.242840 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.244160 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.244421 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.244518 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:03.248240 (+ 3722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:03.248244 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:03.248246 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:03.248274 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:03.250036 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:03.250656 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:03.250659 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:03.250660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:03.250801 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:03.251095 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:03.251098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:03.251099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:03.251352 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:03.251436 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:03.251791 (+ 355us) server_negotiation.cc:299] Negotiation successful
0811 17:22:03.251992 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":87,"server-negotiator.queue_time_us":155}
I20260811 17:22:03.253475 30858 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.188.keytab.
Entry for principal kudu/127.9.157.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.188.keytab.
I20260811 17:22:03.254310 30863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.253977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:32769 (local address 127.0.0.1:41140)
0811 17:22:03.254105 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.254208 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:32769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 17:22:03.254528 30778 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.157.188:32769: connect: Connection refused (error 111)
I20260811 17:22:03.255394 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.188:32769
--webserver_interface=127.9.157.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.188.keytab
--principal=kudu/127.9.157.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:42309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:34917,127.9.157.189:43479,127.9.157.188:32769
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:22:03.255548 30863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.255292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:32769 (local address 127.0.0.1:41148)
0811 17:22:03.255416 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.255467 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:32769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 17:22:03.255836 30858 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.188:32769: Network error: Client connection negotiation failed: client connection to 127.9.157.188:32769: connect: Connection refused (error 111)
I20260811 17:22:03.272855 30742 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } attempt: 1
I20260811 17:22:03.273983 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.273551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:32769 (local address 127.0.0.1:41160)
0811 17:22:03.273750 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.273853 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:32769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 17:22:03.275058 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.274736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:32769 (local address 127.0.0.1:41164)
0811 17:22:03.274916 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.274978 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:32769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260811 17:22:03.275208 30742 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.188:32769: Network error: Client connection negotiation failed: client connection to 127.9.157.188:32769: connect: Connection refused (error 111)
I20260811 17:22:03.305171 30858 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } attempt: 1
I20260811 17:22:03.306674 30863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.305875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:32769 (local address 127.0.0.1:41178)
0811 17:22:03.306361 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.306509 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:32769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
I20260811 17:22:03.308002 30863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.307477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:32769 (local address 127.0.0.1:41194)
0811 17:22:03.307775 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.307865 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:32769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 17:22:03.308246 30858 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.188:32769: Network error: Client connection negotiation failed: client connection to 127.9.157.188:32769: connect: Connection refused (error 111)
I20260811 17:22:03.350411 30742 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } attempt: 2
I20260811 17:22:03.351502 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.351046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:32769 (local address 127.0.0.1:41196)
0811 17:22:03.351266 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.351376 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:32769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 17:22:03.352522 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.352202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:32769 (local address 127.0.0.1:41204)
0811 17:22:03.352350 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.352425 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:32769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 17:22:03.352692 30742 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.188:32769: Network error: Client connection negotiation failed: client connection to 127.9.157.188:32769: connect: Connection refused (error 111)
I20260811 17:22:03.376439 30858 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } attempt: 2
I20260811 17:22:03.377449 30863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.377063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:32769 (local address 127.0.0.1:41206)
0811 17:22:03.377254 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.377348 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:32769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 17:22:03.378211 30863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.377962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:32769 (local address 127.0.0.1:41222)
0811 17:22:03.378088 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.378145 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:32769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 17:22:03.378369 30858 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.188:32769: Network error: Client connection negotiation failed: client connection to 127.9.157.188:32769: connect: Connection refused (error 111)
W20260811 17:22:03.390695 30866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:22:03.391041 30866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:22:03.391088 30866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:22:03.395514 30866 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:22:03.395607 30866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:22:03.395628 30866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:22:03.395709 30866 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:22:03.395761 30866 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:22:03.401649 30866 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:42309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:34917,127.9.157.189:43479,127.9.157.188:32769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.188.keytab
--principal=kudu/127.9.157.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.188:32769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.157.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:22:03.403002 30866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:22:03.404366 30866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:22:03.414127 30875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:03.414213 30866 server_base.cc:1034] running on GCE node
W20260811 17:22:03.414127 30876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:03.414127 30878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:03.415230 30866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:22:03.416635 30866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:22:03.417850 30866 hybrid_clock.cc:648] HybridClock initialized: now 1786468923417826 us; error 51 us; skew 500 ppm
Aug 11 17:22:03 dist-test-slave-mjjr krb5kdc[30622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468923, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:22:03.420994 30866 init.cc:377] Logged in from keytab as kudu/127.9.157.188@KRBTEST.COM (short username kudu)
I20260811 17:22:03.422057 30866 webserver.cc:460] Webserver started at http://127.9.157.188:44113/ using document root <none> and password file <none>
I20260811 17:22:03.422652 30866 fs_manager.cc:359] Metadata directory not provided
I20260811 17:22:03.422719 30866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:22:03.422902 30866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:22:03.425246 30866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-2/data/instance:
uuid: "ad9d7014c2e54d1dbeb7afec872b5f14"
format_stamp: "Formatted at 2026-08-11 17:22:03 on dist-test-slave-mjjr"
server_key: "15ca55da7e87e1081ce6909d49e2a45e"
server_key_iv: "c715dd54d94b48e325f0db3015a839af"
server_key_version: "encryptionkey@0"
I20260811 17:22:03.425875 30866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-2/wal/instance:
uuid: "ad9d7014c2e54d1dbeb7afec872b5f14"
format_stamp: "Formatted at 2026-08-11 17:22:03 on dist-test-slave-mjjr"
server_key: "15ca55da7e87e1081ce6909d49e2a45e"
server_key_iv: "c715dd54d94b48e325f0db3015a839af"
server_key_version: "encryptionkey@0"
I20260811 17:22:03.430122 30866 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:22:03.433223 30887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:03.434736 30866 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:22:03.434927 30866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-2/wal
uuid: "ad9d7014c2e54d1dbeb7afec872b5f14"
format_stamp: "Formatted at 2026-08-11 17:22:03 on dist-test-slave-mjjr"
server_key: "15ca55da7e87e1081ce6909d49e2a45e"
server_key_iv: "c715dd54d94b48e325f0db3015a839af"
server_key_version: "encryptionkey@0"
I20260811 17:22:03.435075 30866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:22:03.451851 30742 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } attempt: 3
I20260811 17:22:03.452865 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.452520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:32769 (local address 127.0.0.1:41230)
0811 17:22:03.452695 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.452770 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:32769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 17:22:03.453933 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.453518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:32769 (local address 127.0.0.1:41240)
0811 17:22:03.453774 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.453835 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:32769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173}
W20260811 17:22:03.454088 30742 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.188:32769: Network error: Client connection negotiation failed: client connection to 127.9.157.188:32769: connect: Connection refused (error 111)
I20260811 17:22:03.462234 30866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:22:03.465611 30866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:22:03.465844 30866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:22:03.469532 30858 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } attempt: 3
I20260811 17:22:03.470636 30863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.470208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:32769 (local address 127.0.0.1:41254)
0811 17:22:03.470464 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.470533 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:32769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
I20260811 17:22:03.471753 30863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.471315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:32769 (local address 127.0.0.1:41270)
0811 17:22:03.471550 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.471615 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:32769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
W20260811 17:22:03.471948 30858 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.188:32769: Network error: Client connection negotiation failed: client connection to 127.9.157.188:32769: connect: Connection refused (error 111)
I20260811 17:22:03.474512 30866 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.188:32769
I20260811 17:22:03.474517 30968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.188:32769 every 8 connection(s)
I20260811 17:22:03.475720 30866 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-2/data/info.pb
I20260811 17:22:03.479082 30969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:22:03.483898 30969 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34917 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:22:03.484714 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 30866
I20260811 17:22:03.484894 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786468743878666-9846-0/minicluster-data/master-2/wal/instance
I20260811 17:22:03.485198 9846 external_mini_cluster.cc:1428] Setting key 3fe07ff054adcb2236ccbab763c88e74
Aug 11 17:22:03 dist-test-slave-mjjr krb5kdc[30622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468923, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.188@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
Aug 11 17:22:03 dist-test-slave-mjjr krb5kdc[30622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.188@KRBTEST.COM
I20260811 17:22:03.501875 30745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.486367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37920 (local address 127.9.157.190:34917)
0811 17:22:03.486523 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:03.486528 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:03.487606 (+ 1078us) server_negotiation.cc:407] Connection header received
0811 17:22:03.488834 (+ 1228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:03.488838 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:03.488980 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:03.489087 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:03.490473 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.491360 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.493124 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.493498 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.493625 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:03.497164 (+ 3539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:03.497172 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:03.497178 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:03.497237 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:03.499167 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:03.499858 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:03.499863 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:03.499865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:03.499972 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:03.500484 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:03.500488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:03.500490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:03.500790 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:03.500871 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:03.501550 (+ 679us) server_negotiation.cc:299] Negotiation successful
0811 17:22:03.501701 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 17:22:03.502938 30972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.486738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34917 (local address 127.0.0.1:37920)
0811 17:22:03.487441 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.487495 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:03.488581 (+ 1086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:03.489220 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:03.489226 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:03.489767 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:22:03.490297 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:03.490325 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.491514 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.491517 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:03.492953 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:03.492967 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.493695 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.493698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:03.493882 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:03.494805 (+ 923us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:22:03.494836 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:22:03.496934 (+ 2098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:22:03.499342 (+ 2408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:03.499352 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:03.499363 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:03.499728 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:03.500088 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:03.500092 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:03.500094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:03.500315 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:03.500929 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:22:03.500939 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:22:03.501260 (+ 321us) client_negotiation.cc:769] Sending connection context
0811 17:22:03.501465 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 17:22:03.501898 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":2,"thread_start_us":288,"threads_started":1}
I20260811 17:22:03.502936 30974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.486739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41276 (local address 127.9.157.188:32769)
0811 17:22:03.487412 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:03.487429 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:03.487466 (+ 37us) server_negotiation.cc:407] Connection header received
0811 17:22:03.488573 (+ 1107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:03.488598 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:03.488954 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:03.489153 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:03.490392 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.491848 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.493259 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.493881 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.493966 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:03.497731 (+ 3765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:03.497742 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:03.497778 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:03.497811 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:03.499939 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:03.500504 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:03.500511 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:03.500513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:03.500597 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:03.500972 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:03.500975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:03.500976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:03.501265 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:03.501443 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:03.501738 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 17:22:03.501965 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":531,"spinlock_wait_cycles":1408,"thread_start_us":192,"threads_started":1}
I20260811 17:22:03.504542 30969 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.189" port: 43479 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 17:22:03 dist-test-slave-mjjr krb5kdc[30622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468923, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.188@KRBTEST.COM for kudu/127.9.157.189@KRBTEST.COM
Aug 11 17:22:03 dist-test-slave-mjjr krb5kdc[30622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:22:03.514359 30972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.505158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:43479 (local address 127.0.0.1:56486)
0811 17:22:03.505286 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.505305 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:03.505408 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:03.505866 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:03.505869 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:03.506107 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:22:03.506248 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:03.506256 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.507262 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.507266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:03.508091 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:03.508101 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.508687 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.508689 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:03.508776 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:03.509307 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:22:03.509324 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:22:03.510847 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:22:03.512746 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:03.512749 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:03.512751 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:03.512988 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:03.513292 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:03.513295 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:03.513297 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:03.513383 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:03.513785 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:22:03.513788 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:22:03.513895 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 17:22:03.514004 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 17:22:03.514169 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 17:22:03.514359 30860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.505452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56486 (local address 127.9.157.189:43479)
0811 17:22:03.505615 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:03.505621 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:03.505634 (+ 13us) server_negotiation.cc:407] Connection header received
0811 17:22:03.505700 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:03.505703 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:03.505755 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:03.505846 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:03.506384 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.507133 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.508222 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.508568 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.508687 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:03.511029 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:03.511033 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:03.511035 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:03.511063 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:03.512613 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:03.513098 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:03.513102 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:03.513103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:03.513189 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:03.513475 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:03.513479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:03.513481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:03.513682 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:03.513788 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:03.514060 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 17:22:03.514177 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 17:22:03.515164 30969 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:22:03.515360 30745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.505682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37926 (local address 127.9.157.190:34917)
0811 17:22:03.505801 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:03.505805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:03.505869 (+ 64us) server_negotiation.cc:407] Connection header received
0811 17:22:03.506006 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:03.506009 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:03.506062 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:03.506146 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:03.507021 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.507798 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.508884 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.509215 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.509324 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:03.511915 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:03.511922 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:03.511925 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:03.511962 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:03.513461 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:03.513922 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:03.513926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:03.513928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:03.514049 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:03.514369 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:03.514373 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:03.514374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:03.514642 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:03.514871 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:03.515052 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 17:22:03.515182 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 17:22:03 dist-test-slave-mjjr krb5kdc[30622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468923, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.188@KRBTEST.COM for kudu/127.9.157.188@KRBTEST.COM
Aug 11 17:22:03 dist-test-slave-mjjr krb5kdc[30622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.189@KRBTEST.COM
I20260811 17:22:03.524616 30972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.515830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:32769 (local address 127.0.0.1:41280)
0811 17:22:03.515942 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.515958 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:03.516045 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:03.516288 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:03.516292 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:03.516465 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:22:03.516546 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:03.516551 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.517435 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.517437 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:03.518249 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:03.518257 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.518709 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.518712 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:03.518794 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:03.519284 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:22:03.519295 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:22:03.520865 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:22:03.522711 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:03.522717 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:03.522721 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:03.523000 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:03.523324 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:03.523330 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:03.523334 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:03.523434 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:03.523948 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:22:03.523954 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:22:03.524068 (+ 114us) client_negotiation.cc:769] Sending connection context
0811 17:22:03.524204 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 17:22:03.524386 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 17:22:03.524699 30974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.515886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41280 (local address 127.9.157.188:32769)
0811 17:22:03.516007 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:03.516011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:03.516025 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:22:03.516128 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:03.516133 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:03.516175 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:03.516247 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:03.516685 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.517343 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.518373 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.518614 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.518693 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:03.521048 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:03.521052 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:03.521053 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:03.521077 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:03.522598 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:03.523110 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:03.523115 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:03.523119 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:03.523220 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:03.523543 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:03.523547 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:03.523550 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:03.523840 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:03.523922 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:03.524244 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 17:22:03.524462 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":87,"server-negotiator.queue_time_us":54}
I20260811 17:22:03.526330 30860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.516666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56496 (local address 127.9.157.189:43479)
0811 17:22:03.516784 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:03.516788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:03.516801 (+ 13us) server_negotiation.cc:407] Connection header received
0811 17:22:03.516835 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:03.516838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:03.516877 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:03.516961 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:03.517503 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.518252 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.519621 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.519981 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.520068 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:03.522632 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:03.522637 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:03.522639 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:03.522674 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:03.524406 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:03.524955 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:03.524958 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:03.524960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:03.525049 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:03.525346 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:03.525348 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:03.525349 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:03.525544 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:03.525733 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:03.526020 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 17:22:03.526168 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 17:22:03.529815 30969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14: Bootstrap starting.
I20260811 17:22:03.529943 9846 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:22:03.533264 30969 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14: Neither blocks nor log segments found. Creating new log.
I20260811 17:22:03.534379 30969 log.cc:824] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14: Log is configured to *not* fsync() on all Append() calls
I20260811 17:22:03.537817 30969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14: No bootstrap required, opened a new log
I20260811 17:22:03.543238 30969 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1aac9a869bf4a4fb6689722529b9a73" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34917 } } peers { permanent_uuid: "61d8814ce09e49e08c225c7b6e88f33d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 43479 } } peers { permanent_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } }
I20260811 17:22:03.543606 30969 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:22:03.543690 30969 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9d7014c2e54d1dbeb7afec872b5f14, State: Initialized, Role: FOLLOWER
I20260811 17:22:03.543882 30974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.532579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41286 (local address 127.9.157.188:32769)
0811 17:22:03.532717 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:03.532722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:03.533197 (+ 475us) server_negotiation.cc:407] Connection header received
0811 17:22:03.533370 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:03.533373 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:03.533426 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:03.533478 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:03.534488 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.535231 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.536653 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.537296 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.537383 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:03.539220 (+ 1837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:03.539223 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:03.539226 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:03.539253 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:03.541379 (+ 2126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:03.542218 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:03.542224 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:03.542228 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:03.542330 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:03.542714 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:03.542721 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:03.542725 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:03.543016 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:03.543118 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:03.543516 (+ 398us) server_negotiation.cc:299] Negotiation successful
0811 17:22:03.543684 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 17:22:03.544016 30745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.532190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37932 (local address 127.9.157.190:34917)
0811 17:22:03.532335 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:03.532340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:03.532584 (+ 244us) server_negotiation.cc:407] Connection header received
0811 17:22:03.532819 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:03.532822 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:03.532881 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:03.532974 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:03.534075 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.534828 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.535903 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.536247 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.536326 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:03.539616 (+ 3290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:03.539621 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:03.539624 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:03.539694 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:03.541699 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:03.542225 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:03.542229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:03.542231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:03.542321 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:03.542640 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:03.542644 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:03.542646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:03.542905 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:03.543008 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:03.543408 (+ 400us) server_negotiation.cc:299] Negotiation successful
0811 17:22:03.543548 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 17:22:03.544443 30969 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1aac9a869bf4a4fb6689722529b9a73" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34917 } } peers { permanent_uuid: "61d8814ce09e49e08c225c7b6e88f33d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 43479 } } peers { permanent_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } }
I20260811 17:22:03.544898 30860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.532301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56508 (local address 127.9.157.189:43479)
0811 17:22:03.532433 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:03.532441 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:03.533248 (+ 807us) server_negotiation.cc:407] Connection header received
0811 17:22:03.533312 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:03.533318 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:03.533371 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:03.533440 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:03.534403 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.535174 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.536495 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.536854 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.536998 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:03.540144 (+ 3146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:03.540149 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:03.540152 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:03.540184 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:03.542841 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:03.543351 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:03.543357 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:03.543360 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:03.543465 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:03.543803 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:03.543807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:03.543809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:03.544066 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:03.544190 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:03.544597 (+ 407us) server_negotiation.cc:299] Negotiation successful
0811 17:22:03.544721 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 17:22:03.545828 30994 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1aac9a869bf4a4fb6689722529b9a73" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34917 } } peers { permanent_uuid: "61d8814ce09e49e08c225c7b6e88f33d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 43479 } } peers { permanent_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } } }
I20260811 17:22:03.545993 30994 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:22:03.546851 30969 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:22:03.550452 30866 master_runner.cc:186] Error getting master registration for 127.9.157.190:34917: OK, instance_id { permanent_uuid: "d1aac9a869bf4a4fb6689722529b9a73" instance_seqno: 1786468922965247 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
W20260811 17:22:03.553115 31010 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260811 17:22:03.553143 30866 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 17:22:03.553247 31010 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:22:03.615264 30858 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } attempt: 4
I20260811 17:22:03.618304 30742 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } attempt: 4
Aug 11 17:22:03 dist-test-slave-mjjr krb5kdc[30622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468923, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.189@KRBTEST.COM for kudu/127.9.157.188@KRBTEST.COM
Aug 11 17:22:03 dist-test-slave-mjjr krb5kdc[30622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468922, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for kudu/127.9.157.188@KRBTEST.COM
I20260811 17:22:03.627545 30863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.616021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:32769 (local address 127.0.0.1:41294)
0811 17:22:03.616187 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.616207 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:03.616333 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:03.616596 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:03.616599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:03.616806 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:22:03.616904 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:03.616909 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.617671 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.617675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:03.618223 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:03.618232 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.618825 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.618830 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:03.618912 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:03.619603 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:22:03.619623 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:22:03.621443 (+ 1820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:22:03.623907 (+ 2464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:03.623911 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:03.623913 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:03.624216 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:03.625620 (+ 1404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:03.625624 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:03.625626 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:03.625760 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:03.626802 (+ 1042us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:22:03.626808 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:22:03.626977 (+ 169us) client_negotiation.cc:769] Sending connection context
0811 17:22:03.627115 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 17:22:03.627320 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 17:22:03.627569 30974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.616149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41294 (local address 127.9.157.188:32769)
0811 17:22:03.616316 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:03.616322 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:03.616339 (+ 17us) server_negotiation.cc:407] Connection header received
0811 17:22:03.616428 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:03.616431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:03.616483 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:03.616572 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:03.617033 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.617553 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.618360 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.618691 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.618817 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:03.621582 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:03.621587 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:03.621589 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:03.621624 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:03.623783 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:03.624344 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:03.624348 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:03.624350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:03.624464 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:03.626061 (+ 1597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:03.626069 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:03.626071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:03.626513 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:03.626863 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:03.627202 (+ 339us) server_negotiation.cc:299] Negotiation successful
0811 17:22:03.627380 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 17:22:03.630131 31013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.619155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41300 (local address 127.9.157.188:32769)
0811 17:22:03.619697 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:03.619705 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:03.619724 (+ 19us) server_negotiation.cc:407] Connection header received
0811 17:22:03.619757 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:03.619759 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:03.619798 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:03.619901 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:03.620337 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.621187 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.622351 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.622705 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.622819 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:03.626001 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:03.626005 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:03.626007 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:03.626034 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:03.627994 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:03.628588 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:03.628593 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:03.628598 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:03.628690 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:03.629040 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:03.629043 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:03.629044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:03.629337 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:03.629448 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:03.629806 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 17:22:03.629942 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":176,"threads_started":1}
I20260811 17:22:03.630326 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:03.618912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:32769 (local address 127.0.0.1:41300)
0811 17:22:03.619150 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:03.619170 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:03.619256 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:03.619926 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:03.619930 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:03.620123 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:22:03.620217 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:03.620223 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.621318 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.621322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:03.622173 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:03.622187 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:03.622873 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:03.622878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:03.622966 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:03.623402 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:22:03.623420 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:22:03.625858 (+ 2438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:22:03.628138 (+ 2280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:03.628144 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:03.628149 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:03.628455 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:03.628794 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:03.628798 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:03.628800 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:03.628920 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:03.629457 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:22:03.629461 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:22:03.629585 (+ 124us) client_negotiation.cc:769] Sending connection context
0811 17:22:03.629713 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 17:22:03.630111 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116}
I20260811 17:22:03.632112 30858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61d8814ce09e49e08c225c7b6e88f33d: Bootstrap starting.
I20260811 17:22:03.634832 30858 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61d8814ce09e49e08c225c7b6e88f33d: Neither blocks nor log segments found. Creating new log.
I20260811 17:22:03.634794 30742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1aac9a869bf4a4fb6689722529b9a73: Bootstrap starting.
I20260811 17:22:03.635735 30858 log.cc:824] T 00000000000000000000000000000000 P 61d8814ce09e49e08c225c7b6e88f33d: Log is configured to *not* fsync() on all Append() calls
I20260811 17:22:03.639549 30858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61d8814ce09e49e08c225c7b6e88f33d: No bootstrap required, opened a new log
I20260811 17:22:03.639714 30742 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1aac9a869bf4a4fb6689722529b9a73: Neither blocks nor log segments found. Creating new log.
I20260811 17:22:03.640694 30742 log.cc:824] T 00000000000000000000000000000000 P d1aac9a869bf4a4fb6689722529b9a73: Log is configured to *not* fsync() on all Append() calls
I20260811 17:22:03.643252 30742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1aac9a869bf4a4fb6689722529b9a73: No bootstrap required, opened a new log
I20260811 17:22:03.643944 30858 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61d8814ce09e49e08c225c7b6e88f33d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1aac9a869bf4a4fb6689722529b9a73" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34917 } } peers { permanent_uuid: "61d8814ce09e49e08c225c7b6e88f33d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 43479 } } peers { permanent_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } }
I20260811 17:22:03.644200 30858 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61d8814ce09e49e08c225c7b6e88f33d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:22:03.644274 30858 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61d8814ce09e49e08c225c7b6e88f33d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d8814ce09e49e08c225c7b6e88f33d, State: Initialized, Role: FOLLOWER
I20260811 17:22:03.644882 30858 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61d8814ce09e49e08c225c7b6e88f33d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1aac9a869bf4a4fb6689722529b9a73" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34917 } } peers { permanent_uuid: "61d8814ce09e49e08c225c7b6e88f33d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 43479 } } peers { permanent_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } }
I20260811 17:22:03.645561 31016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61d8814ce09e49e08c225c7b6e88f33d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1aac9a869bf4a4fb6689722529b9a73" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34917 } } peers { permanent_uuid: "61d8814ce09e49e08c225c7b6e88f33d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 43479 } } peers { permanent_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } } }
I20260811 17:22:03.645782 31016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61d8814ce09e49e08c225c7b6e88f33d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:22:03.646627 30858 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61d8814ce09e49e08c225c7b6e88f33d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:22:03.647673 30742 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1aac9a869bf4a4fb6689722529b9a73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1aac9a869bf4a4fb6689722529b9a73" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34917 } } peers { permanent_uuid: "61d8814ce09e49e08c225c7b6e88f33d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 43479 } } peers { permanent_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } }
I20260811 17:22:03.647953 30742 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1aac9a869bf4a4fb6689722529b9a73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:22:03.648032 30742 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1aac9a869bf4a4fb6689722529b9a73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1aac9a869bf4a4fb6689722529b9a73, State: Initialized, Role: FOLLOWER
I20260811 17:22:03.648589 30742 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1aac9a869bf4a4fb6689722529b9a73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1aac9a869bf4a4fb6689722529b9a73" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34917 } } peers { permanent_uuid: "61d8814ce09e49e08c225c7b6e88f33d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 43479 } } peers { permanent_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } }
I20260811 17:22:03.650396 30742 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1aac9a869bf4a4fb6689722529b9a73 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:22:03.651880 30994 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:22:03.652100 30994 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1aac9a869bf4a4fb6689722529b9a73" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34917 } } peers { permanent_uuid: "61d8814ce09e49e08c225c7b6e88f33d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 43479 } } peers { permanent_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } }
I20260811 17:22:03.653424 30994 leader_election.cc:288] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1aac9a869bf4a4fb6689722529b9a73 (127.9.157.190:34917), 61d8814ce09e49e08c225c7b6e88f33d (127.9.157.189:43479)
W20260811 17:22:03.653662 31033 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61d8814ce09e49e08c225c7b6e88f33d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260811 17:22:03.654361 30704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1aac9a869bf4a4fb6689722529b9a73" is_pre_election: true
I20260811 17:22:03.654692 30704 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d1aac9a869bf4a4fb6689722529b9a73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad9d7014c2e54d1dbeb7afec872b5f14 in term 0.
I20260811 17:22:03.655382 30894 leader_election.cc:302] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad9d7014c2e54d1dbeb7afec872b5f14, d1aac9a869bf4a4fb6689722529b9a73; no voters:
I20260811 17:22:03.655845 30994 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:22:03.655956 30994 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:22:03.656020 30994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 [term 0 FOLLOWER]: Advancing to term 1
W20260811 17:22:03.657755 31050 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d1aac9a869bf4a4fb6689722529b9a73: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260811 17:22:03.657683 30994 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1aac9a869bf4a4fb6689722529b9a73" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34917 } } peers { permanent_uuid: "61d8814ce09e49e08c225c7b6e88f33d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 43479 } } peers { permanent_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } }
W20260811 17:22:03.657866 31050 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:22:03.650461 30751 master_runner.cc:186] Error getting master registration for 127.9.157.190:34917: OK, instance_id { permanent_uuid: "d1aac9a869bf4a4fb6689722529b9a73" instance_seqno: 1786468922965247 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
W20260811 17:22:03.658012 31033 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:22:03.658167 30751 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 17:22:03.654397 30817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d8814ce09e49e08c225c7b6e88f33d" is_pre_election: true
I20260811 17:22:03.657819 31036 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1aac9a869bf4a4fb6689722529b9a73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1aac9a869bf4a4fb6689722529b9a73" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34917 } } peers { permanent_uuid: "61d8814ce09e49e08c225c7b6e88f33d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 43479 } } peers { permanent_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } } }
I20260811 17:22:03.658427 30817 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 61d8814ce09e49e08c225c7b6e88f33d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad9d7014c2e54d1dbeb7afec872b5f14 in term 0.
I20260811 17:22:03.658461 31036 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1aac9a869bf4a4fb6689722529b9a73 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:22:03.658720 30814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61d8814ce09e49e08c225c7b6e88f33d"
I20260811 17:22:03.658883 30814 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61d8814ce09e49e08c225c7b6e88f33d [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:22:03.660406 30814 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 61d8814ce09e49e08c225c7b6e88f33d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad9d7014c2e54d1dbeb7afec872b5f14 in term 1.
I20260811 17:22:03.660579 30704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1aac9a869bf4a4fb6689722529b9a73"
I20260811 17:22:03.660744 30704 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1aac9a869bf4a4fb6689722529b9a73 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:22:03.661855 30890 leader_election.cc:302] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61d8814ce09e49e08c225c7b6e88f33d, ad9d7014c2e54d1dbeb7afec872b5f14; no voters:
I20260811 17:22:03.662266 31052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:22:03.662324 30704 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d1aac9a869bf4a4fb6689722529b9a73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad9d7014c2e54d1dbeb7afec872b5f14 in term 1.
I20260811 17:22:03.662528 31052 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 [term 1 LEADER]: Becoming Leader. State: Replica: ad9d7014c2e54d1dbeb7afec872b5f14, State: Running, Role: LEADER
I20260811 17:22:03.662914 31052 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1aac9a869bf4a4fb6689722529b9a73" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34917 } } peers { permanent_uuid: "61d8814ce09e49e08c225c7b6e88f33d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 43479 } } peers { permanent_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } }
I20260811 17:22:03.663003 30994 leader_election.cc:288] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 [CANDIDATE]: Term 1 election: Requested vote from peers d1aac9a869bf4a4fb6689722529b9a73 (127.9.157.190:34917), 61d8814ce09e49e08c225c7b6e88f33d (127.9.157.189:43479)
I20260811 17:22:03.666467 30994 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad9d7014c2e54d1dbeb7afec872b5f14. Latest consensus state: current_term: 1 leader_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1aac9a869bf4a4fb6689722529b9a73" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34917 } } peers { permanent_uuid: "61d8814ce09e49e08c225c7b6e88f33d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 43479 } } peers { permanent_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } } }
I20260811 17:22:03.666615 30994 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 [sys.catalog]: This master's current role is: LEADER
I20260811 17:22:03.667630 30644 master_runner.cc:232] Existing masters have differing terms: 0 vs 1
I20260811 17:22:03.667822 30644 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 17:22:03.675813 31055 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:22:03.677091 31055 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:22:03.688576 30814 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 61d8814ce09e49e08c225c7b6e88f33d [term 1 FOLLOWER]: Refusing update from remote peer ad9d7014c2e54d1dbeb7afec872b5f14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:22:03.689417 31053 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61d8814ce09e49e08c225c7b6e88f33d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 43479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:22:03.694329 31016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61d8814ce09e49e08c225c7b6e88f33d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad9d7014c2e54d1dbeb7afec872b5f14. Latest consensus state: current_term: 1 leader_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1aac9a869bf4a4fb6689722529b9a73" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34917 } } peers { permanent_uuid: "61d8814ce09e49e08c225c7b6e88f33d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 43479 } } peers { permanent_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } } }
I20260811 17:22:03.694505 31016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61d8814ce09e49e08c225c7b6e88f33d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:22:03.695210 30704 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d1aac9a869bf4a4fb6689722529b9a73 [term 1 FOLLOWER]: Refusing update from remote peer ad9d7014c2e54d1dbeb7afec872b5f14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:22:03.695917 31053 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1aac9a869bf4a4fb6689722529b9a73" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:22:03.700325 31036 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1aac9a869bf4a4fb6689722529b9a73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad9d7014c2e54d1dbeb7afec872b5f14. Latest consensus state: current_term: 1 leader_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1aac9a869bf4a4fb6689722529b9a73" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34917 } } peers { permanent_uuid: "61d8814ce09e49e08c225c7b6e88f33d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 43479 } } peers { permanent_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } } }
I20260811 17:22:03.700488 31036 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1aac9a869bf4a4fb6689722529b9a73 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:22:03.701913 31053 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1aac9a869bf4a4fb6689722529b9a73" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34917 } } peers { permanent_uuid: "61d8814ce09e49e08c225c7b6e88f33d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 43479 } } peers { permanent_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } } }
W20260811 17:22:03.701990 30782 tablet.cc:2369] T 00000000000000000000000000000000 P 61d8814ce09e49e08c225c7b6e88f33d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:22:03.702059 31053 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 [sys.catalog]: This master's current role is: LEADER
I20260811 17:22:03.703450 31016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61d8814ce09e49e08c225c7b6e88f33d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1aac9a869bf4a4fb6689722529b9a73" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34917 } } peers { permanent_uuid: "61d8814ce09e49e08c225c7b6e88f33d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 43479 } } peers { permanent_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } } }
I20260811 17:22:03.703583 31016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61d8814ce09e49e08c225c7b6e88f33d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:22:03.703730 31055 catalog_manager.cc:1347] Generated new cluster ID: 25ca70d4d2cd43bcb34624bc24b77bf1
I20260811 17:22:03.703788 31055 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260811 17:22:03.711596 30672 tablet.cc:2369] T 00000000000000000000000000000000 P d1aac9a869bf4a4fb6689722529b9a73: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:22:03.712317 31060 mvcc.cc:205] Tried to move back new op lower bound from 7317376711416664064 to 7317376711327137792. Current Snapshot: MvccSnapshot[applied={T|T < 7317376711416664064}]
I20260811 17:22:03.719282 30994 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1aac9a869bf4a4fb6689722529b9a73" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34917 } } peers { permanent_uuid: "61d8814ce09e49e08c225c7b6e88f33d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 43479 } } peers { permanent_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } } }
I20260811 17:22:03.719453 30994 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 [sys.catalog]: This master's current role is: LEADER
I20260811 17:22:03.721304 31036 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1aac9a869bf4a4fb6689722529b9a73 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1aac9a869bf4a4fb6689722529b9a73" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34917 } } peers { permanent_uuid: "61d8814ce09e49e08c225c7b6e88f33d" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 43479 } } peers { permanent_uuid: "ad9d7014c2e54d1dbeb7afec872b5f14" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 32769 } } }
I20260811 17:22:03.721460 31036 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1aac9a869bf4a4fb6689722529b9a73 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:22:03.722841 31062 mvcc.cc:205] Tried to move back new op lower bound from 7317376711416664064 to 7317376711327137792. Current Snapshot: MvccSnapshot[applied={T|T < 7317376711416664064}]
I20260811 17:22:03.739043 31055 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:22:03.743535 31055 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:22:03.757237 31055 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14: Generated new TSK 0
I20260811 17:22:03.758298 31055 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:22:04.659333 31050 catalog_manager.cc:1259] Loaded cluster ID: 25ca70d4d2cd43bcb34624bc24b77bf1
I20260811 17:22:04.659363 31033 catalog_manager.cc:1259] Loaded cluster ID: 25ca70d4d2cd43bcb34624bc24b77bf1
I20260811 17:22:04.659461 31033 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 61d8814ce09e49e08c225c7b6e88f33d: loading cluster ID for follower catalog manager: success
I20260811 17:22:04.659464 31050 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d1aac9a869bf4a4fb6689722529b9a73: loading cluster ID for follower catalog manager: success
I20260811 17:22:04.661247 31050 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d1aac9a869bf4a4fb6689722529b9a73: acquiring CA information for follower catalog manager: success
I20260811 17:22:04.661247 31033 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 61d8814ce09e49e08c225c7b6e88f33d: acquiring CA information for follower catalog manager: success
I20260811 17:22:04.662511 31050 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d1aac9a869bf4a4fb6689722529b9a73: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 17:22:04.662511 31033 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 61d8814ce09e49e08c225c7b6e88f33d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 17:22:04.827246 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 30644
W20260811 17:22:04.848975 30894 connection.cc:535] client connection to 127.9.157.190:34917 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 17:22:04.849251 30894 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 17:22:04.849403 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 30751
I20260811 17:22:04.856076 31082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:04.854555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34917 (local address 127.0.0.1:37948)
0811 17:22:04.855821 (+ 1266us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:04.855930 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1150,"thread_start_us":991,"threads_started":1}
W20260811 17:22:04.856245 30894 consensus_peers.cc:487] T 00000000000000000000000000000000 P ad9d7014c2e54d1dbeb7afec872b5f14 -> Peer d1aac9a869bf4a4fb6689722529b9a73 (127.9.157.190:34917): Couldn't send request to peer d1aac9a869bf4a4fb6689722529b9a73. Status: Network error: Client connection negotiation failed: client connection to 127.9.157.190:34917: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 17:22:04.865557 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 30866
2026-08-11T17:22:04Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3546 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:22:04 dist-test-slave-mjjr krb5kdc[31088](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:22:04 dist-test-slave-mjjr krb5kdc[31088](info): set up 2 sockets
Aug 11 17:22:04 dist-test-slave-mjjr krb5kdc[31088](info): commencing operation
krb5kdc: starting...
W20260811 17:22:06.155201 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:22:06 dist-test-slave-mjjr krb5kdc[31088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:22:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:22:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:22:06.295331 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:33601
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:39499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:33601
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:22:06.419009 31105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:22:06.419400 31105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:22:06.419490 31105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:22:06.423767 31105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:22:06.423893 31105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:22:06.423931 31105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:22:06.424001 31105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:22:06.424058 31105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:22:06.429965 31105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:39499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:33601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:33601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:22:06.431356 31105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:22:06.432566 31105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:22:06.442353 31111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:06.442423 31116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:06.442368 31113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:06.442785 31105 server_base.cc:1034] running on GCE node
I20260811 17:22:06.444233 31105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:22:06.445600 31105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:22:06.446841 31105 hybrid_clock.cc:648] HybridClock initialized: now 1786468926446806 us; error 53 us; skew 500 ppm
Aug 11 17:22:06 dist-test-slave-mjjr krb5kdc[31088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468926, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:22:06.455185 31105 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:22:06.456519 31105 webserver.cc:460] Webserver started at http://127.9.157.190:43039/ using document root <none> and password file <none>
I20260811 17:22:06.457212 31105 fs_manager.cc:359] Metadata directory not provided
I20260811 17:22:06.457319 31105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:22:06.457592 31105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:22:06.460098 31105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "2df5b414ae7849d3b1672ac13141913d"
format_stamp: "Formatted at 2026-08-11 17:22:06 on dist-test-slave-mjjr"
server_key: "aabd4b483045eb7b39de8a84258a21dd"
server_key_iv: "4f03dcd0cedc8b0faaff377c08f3e313"
server_key_version: "encryptionkey@0"
I20260811 17:22:06.460788 31105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "2df5b414ae7849d3b1672ac13141913d"
format_stamp: "Formatted at 2026-08-11 17:22:06 on dist-test-slave-mjjr"
server_key: "aabd4b483045eb7b39de8a84258a21dd"
server_key_iv: "4f03dcd0cedc8b0faaff377c08f3e313"
server_key_version: "encryptionkey@0"
I20260811 17:22:06.465685 31105 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 17:22:06.468997 31130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:06.470700 31105 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 17:22:06.470942 31105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "2df5b414ae7849d3b1672ac13141913d"
format_stamp: "Formatted at 2026-08-11 17:22:06 on dist-test-slave-mjjr"
server_key: "aabd4b483045eb7b39de8a84258a21dd"
server_key_iv: "4f03dcd0cedc8b0faaff377c08f3e313"
server_key_version: "encryptionkey@0"
I20260811 17:22:06.471100 31105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:22:06.485993 31105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:22:06.489854 31105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:22:06.490096 31105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:22:06.498602 31105 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:33601
I20260811 17:22:06.498701 31210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:33601 every 8 connection(s)
I20260811 17:22:06.499783 31105 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:22:06.503000 31211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:22:06.504436 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 31105
I20260811 17:22:06.504581 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:22:06.504951 9846 external_mini_cluster.cc:1428] Setting key 809761621a6fc15113f4a0ae0fa00bf7
I20260811 17:22:06.511221 31211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2df5b414ae7849d3b1672ac13141913d: Bootstrap starting.
I20260811 17:22:06.514439 31211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2df5b414ae7849d3b1672ac13141913d: Neither blocks nor log segments found. Creating new log.
Aug 11 17:22:06 dist-test-slave-mjjr krb5kdc[31088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:22:06.515894 31211 log.cc:824] T 00000000000000000000000000000000 P 2df5b414ae7849d3b1672ac13141913d: Log is configured to *not* fsync() on all Append() calls
I20260811 17:22:06.518546 31211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2df5b414ae7849d3b1672ac13141913d: No bootstrap required, opened a new log
I20260811 17:22:06.522461 31214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:06.506679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40810 (local address 127.9.157.190:33601)
0811 17:22:06.507232 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:06.507249 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:06.507300 (+ 51us) server_negotiation.cc:407] Connection header received
0811 17:22:06.508395 (+ 1095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:06.508420 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:06.508804 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:06.509191 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:06.510110 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:06.511129 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:06.512154 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:06.512769 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:06.512960 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:06.516586 (+ 3626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:06.516593 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:06.516612 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:06.516646 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:06.519134 (+ 2488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:06.519724 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:06.519737 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:06.519744 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:06.519877 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:06.520196 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:06.520200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:06.520201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:06.520672 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:06.520901 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:06.521237 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 17:22:06.521595 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":164,"threads_started":1}
I20260811 17:22:06.522944 31211 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2df5b414ae7849d3b1672ac13141913d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2df5b414ae7849d3b1672ac13141913d" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 33601 } }
I20260811 17:22:06.523252 31211 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2df5b414ae7849d3b1672ac13141913d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:22:06.523327 31211 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2df5b414ae7849d3b1672ac13141913d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2df5b414ae7849d3b1672ac13141913d, State: Initialized, Role: FOLLOWER
I20260811 17:22:06.523916 31211 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2df5b414ae7849d3b1672ac13141913d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2df5b414ae7849d3b1672ac13141913d" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 33601 } }
I20260811 17:22:06.524115 31211 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2df5b414ae7849d3b1672ac13141913d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:22:06.524214 31211 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2df5b414ae7849d3b1672ac13141913d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:22:06.524354 31211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2df5b414ae7849d3b1672ac13141913d [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:22:06.525760 31211 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2df5b414ae7849d3b1672ac13141913d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2df5b414ae7849d3b1672ac13141913d" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 33601 } }
I20260811 17:22:06.526173 31211 leader_election.cc:302] T 00000000000000000000000000000000 P 2df5b414ae7849d3b1672ac13141913d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2df5b414ae7849d3b1672ac13141913d; no voters:
I20260811 17:22:06.526547 9846 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:22:06.526683 31211 leader_election.cc:288] T 00000000000000000000000000000000 P 2df5b414ae7849d3b1672ac13141913d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:22:06.526803 31219 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2df5b414ae7849d3b1672ac13141913d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:22:06.527024 31219 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2df5b414ae7849d3b1672ac13141913d [term 1 LEADER]: Becoming Leader. State: Replica: 2df5b414ae7849d3b1672ac13141913d, State: Running, Role: LEADER
I20260811 17:22:06.527382 31219 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2df5b414ae7849d3b1672ac13141913d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2df5b414ae7849d3b1672ac13141913d" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 33601 } }
I20260811 17:22:06.528347 31211 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2df5b414ae7849d3b1672ac13141913d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:22:06.528841 31219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2df5b414ae7849d3b1672ac13141913d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2df5b414ae7849d3b1672ac13141913d. Latest consensus state: current_term: 1 leader_uuid: "2df5b414ae7849d3b1672ac13141913d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2df5b414ae7849d3b1672ac13141913d" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 33601 } } }
I20260811 17:22:06.529210 31219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2df5b414ae7849d3b1672ac13141913d [sys.catalog]: This master's current role is: LEADER
I20260811 17:22:06.529287 31220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2df5b414ae7849d3b1672ac13141913d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2df5b414ae7849d3b1672ac13141913d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2df5b414ae7849d3b1672ac13141913d" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 33601 } } }
I20260811 17:22:06.529410 31220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2df5b414ae7849d3b1672ac13141913d [sys.catalog]: This master's current role is: LEADER
I20260811 17:22:06.529709 31225 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:22:06.533674 31225 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:22:06.544226 31225 catalog_manager.cc:1347] Generated new cluster ID: 7bf89198d0ef481d8b5fb13714114fb2
I20260811 17:22:06.544346 31225 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:22:06.552205 31225 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:22:06.553385 31225 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:22:06.572492 31225 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2df5b414ae7849d3b1672ac13141913d: Generated new TSK 0
I20260811 17:22:06.573407 31225 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:22:06.584097 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 31105
2026-08-11T17:22:06Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1720 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:22:06 dist-test-slave-mjjr krb5kdc[31253](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:22:06 dist-test-slave-mjjr krb5kdc[31253](info): set up 2 sockets
Aug 11 17:22:06 dist-test-slave-mjjr krb5kdc[31253](info): commencing operation
krb5kdc: starting...
W20260811 17:22:07.874099 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.005s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:22:07 dist-test-slave-mjjr krb5kdc[31253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:22:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:22:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:22:08.019017 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:44901
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:44901
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:22:08.149246 31272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:22:08.149649 31272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:22:08.149766 31272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:22:08.156337 31272 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:22:08.156469 31272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:22:08.156507 31272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:22:08.156548 31272 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:22:08.156605 31272 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:22:08.163051 31272 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:44901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:44901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:22:08.164475 31272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:22:08.165699 31272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:22:08.176753 31277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:08.177032 31272 server_base.cc:1034] running on GCE node
W20260811 17:22:08.176822 31281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:08.176822 31283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:08.177927 31272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:22:08.179268 31272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:22:08.180491 31272 hybrid_clock.cc:648] HybridClock initialized: now 1786468928180446 us; error 61 us; skew 500 ppm
Aug 11 17:22:08 dist-test-slave-mjjr krb5kdc[31253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468928, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:22:08.184741 31272 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:22:08.186656 31272 webserver.cc:460] Webserver started at http://127.9.157.190:37181/ using document root <none> and password file <none>
I20260811 17:22:08.187603 31272 fs_manager.cc:359] Metadata directory not provided
I20260811 17:22:08.187804 31272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:22:08.188055 31272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:22:08.190515 31272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0"
format_stamp: "Formatted at 2026-08-11 17:22:08 on dist-test-slave-mjjr"
server_key: "b1e218ce3e3d12af42b137bf90cf6fc5"
server_key_iv: "89e7f2fe7ff6fcfcdd4e86584a1b38a6"
server_key_version: "encryptionkey@0"
I20260811 17:22:08.191136 31272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0"
format_stamp: "Formatted at 2026-08-11 17:22:08 on dist-test-slave-mjjr"
server_key: "b1e218ce3e3d12af42b137bf90cf6fc5"
server_key_iv: "89e7f2fe7ff6fcfcdd4e86584a1b38a6"
server_key_version: "encryptionkey@0"
I20260811 17:22:08.196086 31272 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 17:22:08.199330 31297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:08.201022 31272 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:22:08.201233 31272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0"
format_stamp: "Formatted at 2026-08-11 17:22:08 on dist-test-slave-mjjr"
server_key: "b1e218ce3e3d12af42b137bf90cf6fc5"
server_key_iv: "89e7f2fe7ff6fcfcdd4e86584a1b38a6"
server_key_version: "encryptionkey@0"
I20260811 17:22:08.201400 31272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:22:08.220142 31272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:22:08.223973 31272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:22:08.224224 31272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:22:08.233436 31272 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:44901
I20260811 17:22:08.233832 31375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:44901 every 8 connection(s)
I20260811 17:22:08.234618 31272 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:22:08.238185 31377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:22:08.238540 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 31272
I20260811 17:22:08.238653 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:22:08.238943 9846 external_mini_cluster.cc:1428] Setting key 9bc832e414173885689b1d95bae545ef
I20260811 17:22:08.246374 31377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0: Bootstrap starting.
I20260811 17:22:08.249423 31377 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0: Neither blocks nor log segments found. Creating new log.
Aug 11 17:22:08 dist-test-slave-mjjr krb5kdc[31253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:22:08.250428 31377 log.cc:824] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0: Log is configured to *not* fsync() on all Append() calls
I20260811 17:22:08.253959 31377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0: No bootstrap required, opened a new log
I20260811 17:22:08.257876 31382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:08.240657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42086 (local address 127.9.157.190:44901)
0811 17:22:08.241230 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:08.241248 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:08.241288 (+ 40us) server_negotiation.cc:407] Connection header received
0811 17:22:08.242027 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:08.242048 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:08.242428 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:08.242835 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:08.244021 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:08.245506 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:08.246620 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:08.247152 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:08.247353 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:08.250907 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:08.250919 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:08.250945 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:08.250989 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:08.253607 (+ 2618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:08.254382 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:08.254392 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:08.254403 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:08.254576 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:08.255002 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:08.255011 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:08.255013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:08.255625 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:08.255904 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:08.256241 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 17:22:08.256639 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":191,"threads_started":1}
I20260811 17:22:08.259354 31377 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44901 } }
I20260811 17:22:08.259624 31377 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:22:08.259744 31377 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab1d7cfe1fe9459b9e20d6139e27a4b0, State: Initialized, Role: FOLLOWER
I20260811 17:22:08.260281 31377 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44901 } }
I20260811 17:22:08.260452 31377 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:22:08.260537 31377 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:22:08.260689 31377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:22:08.261811 31377 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44901 } }
I20260811 17:22:08.262220 31377 leader_election.cc:302] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab1d7cfe1fe9459b9e20d6139e27a4b0; no voters:
I20260811 17:22:08.262598 31377 leader_election.cc:288] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:22:08.262769 31384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:22:08.263046 31384 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [term 1 LEADER]: Becoming Leader. State: Replica: ab1d7cfe1fe9459b9e20d6139e27a4b0, State: Running, Role: LEADER
I20260811 17:22:08.263409 31384 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44901 } }
I20260811 17:22:08.264159 31377 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:22:08.265527 31386 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab1d7cfe1fe9459b9e20d6139e27a4b0. Latest consensus state: current_term: 1 leader_uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44901 } } }
I20260811 17:22:08.265720 31386 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [sys.catalog]: This master's current role is: LEADER
I20260811 17:22:08.265527 31385 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44901 } } }
I20260811 17:22:08.265875 31385 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [sys.catalog]: This master's current role is: LEADER
I20260811 17:22:08.266291 31398 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:22:08.270792 31398 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:22:08.277175 31398 catalog_manager.cc:1347] Generated new cluster ID: 74fa366ef4d14041a98b9ee9435ddc01
I20260811 17:22:08.277299 31398 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:22:08.292675 31398 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:22:08.294634 31398 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:22:08.306121 31398 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0: Generated new TSK 0
I20260811 17:22:08.307255 31398 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
I20260811 17:22:08.347550 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:0
--local_ip_for_outbound_sockets=127.9.157.129
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:44901
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:22:08.477223 31417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:22:08.477573 31417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:22:08.477655 31417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:22:08.482048 31417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:22:08.482172 31417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:22:08.482277 31417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:22:08.488161 31417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:44901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:22:08.489487 31417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:22:08.490653 31417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:22:08.501695 31425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:08.501695 31427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:08.501698 31424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:08.502058 31417 server_base.cc:1034] running on GCE node
I20260811 17:22:08.502580 31417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:22:08.503355 31417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:22:08.504537 31417 hybrid_clock.cc:648] HybridClock initialized: now 1786468928504504 us; error 49 us; skew 500 ppm
Aug 11 17:22:08 dist-test-slave-mjjr krb5kdc[31253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468928, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:22:08.507854 31417 init.cc:377] Logged in from keytab as kudu/127.9.157.129@KRBTEST.COM (short username kudu)
I20260811 17:22:08.509024 31417 webserver.cc:460] Webserver started at http://127.9.157.129:33223/ using document root <none> and password file <none>
I20260811 17:22:08.509625 31417 fs_manager.cc:359] Metadata directory not provided
I20260811 17:22:08.509692 31417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:22:08.509876 31417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:22:08.512671 31417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/data/instance:
uuid: "6308b38e7b554a12861b5e97ab5c0aa3"
format_stamp: "Formatted at 2026-08-11 17:22:08 on dist-test-slave-mjjr"
server_key: "714c6f8b86c4ddec405bdc59cbdb79aa"
server_key_iv: "d9bfd631d23c0920869b56b086a46857"
server_key_version: "encryptionkey@0"
I20260811 17:22:08.513281 31417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance:
uuid: "6308b38e7b554a12861b5e97ab5c0aa3"
format_stamp: "Formatted at 2026-08-11 17:22:08 on dist-test-slave-mjjr"
server_key: "714c6f8b86c4ddec405bdc59cbdb79aa"
server_key_iv: "d9bfd631d23c0920869b56b086a46857"
server_key_version: "encryptionkey@0"
I20260811 17:22:08.517541 31417 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:22:08.520525 31438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:08.521998 31417 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:22:08.522169 31417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "6308b38e7b554a12861b5e97ab5c0aa3"
format_stamp: "Formatted at 2026-08-11 17:22:08 on dist-test-slave-mjjr"
server_key: "714c6f8b86c4ddec405bdc59cbdb79aa"
server_key_iv: "d9bfd631d23c0920869b56b086a46857"
server_key_version: "encryptionkey@0"
I20260811 17:22:08.522311 31417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:22:08.543851 31417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:22:08.547569 31417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:22:08.547823 31417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:22:08.548491 31417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:22:08.549430 31417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:22:08.549500 31417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:08.549571 31417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:22:08.549615 31417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:08.559576 31417 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:32955
I20260811 17:22:08.559621 31608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:32955 every 8 connection(s)
I20260811 17:22:08.560844 31417 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:22:08.567521 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 31417
I20260811 17:22:08.567711 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance
I20260811 17:22:08.568064 9846 external_mini_cluster.cc:1428] Setting key 5b6645a1aceef7c66a71f673e1f15380
Aug 11 17:22:08 dist-test-slave-mjjr krb5kdc[31253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468928, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:22:08.579205 31382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:08.563118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:60797 (local address 127.9.157.190:44901)
0811 17:22:08.563309 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:08.563314 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:08.564393 (+ 1079us) server_negotiation.cc:407] Connection header received
0811 17:22:08.566071 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:08.566075 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:08.566135 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:08.566218 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:08.567908 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:08.568545 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:08.570243 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:08.570477 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:08.570563 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:08.574325 (+ 3762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:08.574329 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:08.574331 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:08.574362 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:08.576038 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:08.576963 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:08.576967 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:08.576968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:08.577077 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:08.577650 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:08.577654 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:08.577655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:08.577905 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:08.577993 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:08.578911 (+ 918us) server_negotiation.cc:299] Negotiation successful
0811 17:22:08.579064 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 17:22:08.580763 31615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:08.563538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.129:60797)
0811 17:22:08.564208 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:08.564260 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:08.565727 (+ 1467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:08.566414 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:08.566422 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:08.566882 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:22:08.567606 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:08.567635 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:08.568784 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:08.568788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:08.570110 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:08.570123 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:08.570659 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:08.570666 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:08.570942 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:08.571680 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:22:08.571710 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:22:08.574079 (+ 2369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:22:08.576210 (+ 2131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:08.576222 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:08.576248 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:08.576767 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:08.577202 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:08.577210 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:08.577215 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:08.577487 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:08.578041 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:22:08.578054 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:22:08.578519 (+ 465us) client_negotiation.cc:769] Sending connection context
0811 17:22:08.578816 (+ 297us) client_negotiation.cc:240] Negotiation successful
0811 17:22:08.579300 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":203,"threads_started":1}
WARNING: no policy specified for kudu/127.9.157.130@KRBTEST.COM; defaulting to no policy
I20260811 17:22:08.582487 31612 heartbeater.cc:347] Connected to a master server at 127.9.157.190:44901
I20260811 17:22:08.582808 31612 heartbeater.cc:464] Registering TS with master...
I20260811 17:22:08.583420 31612 heartbeater.cc:511] Master 127.9.157.190:44901 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.130@KRBTEST.COM" created.
I20260811 17:22:08.585371 31320 ts_manager.cc:194] Registered new tserver with Master: 6308b38e7b554a12861b5e97ab5c0aa3 (127.9.157.129:32955)
I20260811 17:22:08.586941 31320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.129@KRBTEST.COM'} at 127.9.157.129:60797
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
I20260811 17:22:08.603191 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:0
--local_ip_for_outbound_sockets=127.9.157.130
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:44901
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:22:08.733577 31621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:22:08.733922 31621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:22:08.733971 31621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:22:08.739075 31621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:22:08.739183 31621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:22:08.739286 31621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:22:08.745384 31621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:44901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:22:08.746762 31621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:22:08.748009 31621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:22:08.759473 31631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:08.759475 31636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:08.759475 31633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:08.759537 31621 server_base.cc:1034] running on GCE node
I20260811 17:22:08.760387 31621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:22:08.761174 31621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:22:08.762405 31621 hybrid_clock.cc:648] HybridClock initialized: now 1786468928762384 us; error 35 us; skew 500 ppm
Aug 11 17:22:08 dist-test-slave-mjjr krb5kdc[31253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468928, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:22:08.766053 31621 init.cc:377] Logged in from keytab as kudu/127.9.157.130@KRBTEST.COM (short username kudu)
I20260811 17:22:08.767328 31621 webserver.cc:460] Webserver started at http://127.9.157.130:45189/ using document root <none> and password file <none>
I20260811 17:22:08.768064 31621 fs_manager.cc:359] Metadata directory not provided
I20260811 17:22:08.768138 31621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:22:08.768395 31621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:22:08.770735 31621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/data/instance:
uuid: "08aa841a1f38462ea7ab7c8fe54d9de0"
format_stamp: "Formatted at 2026-08-11 17:22:08 on dist-test-slave-mjjr"
server_key: "751ba72ecdfdef3dc7e873d35f9a9221"
server_key_iv: "f00062308485ddfbca68230f5b94525f"
server_key_version: "encryptionkey@0"
I20260811 17:22:08.771389 31621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance:
uuid: "08aa841a1f38462ea7ab7c8fe54d9de0"
format_stamp: "Formatted at 2026-08-11 17:22:08 on dist-test-slave-mjjr"
server_key: "751ba72ecdfdef3dc7e873d35f9a9221"
server_key_iv: "f00062308485ddfbca68230f5b94525f"
server_key_version: "encryptionkey@0"
I20260811 17:22:08.776290 31621 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260811 17:22:08.779322 31645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:08.781065 31621 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:22:08.781262 31621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "08aa841a1f38462ea7ab7c8fe54d9de0"
format_stamp: "Formatted at 2026-08-11 17:22:08 on dist-test-slave-mjjr"
server_key: "751ba72ecdfdef3dc7e873d35f9a9221"
server_key_iv: "f00062308485ddfbca68230f5b94525f"
server_key_version: "encryptionkey@0"
I20260811 17:22:08.781414 31621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:22:08.821771 31621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:22:08.825737 31621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:22:08.825971 31621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:22:08.826804 31621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:22:08.827906 31621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:22:08.827970 31621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:08.828017 31621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:22:08.828066 31621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:08.838687 31621 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:39529
I20260811 17:22:08.838719 31810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:39529 every 8 connection(s)
I20260811 17:22:08.839804 31621 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:22:08.843954 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 31621
I20260811 17:22:08.844108 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance
I20260811 17:22:08.844395 9846 external_mini_cluster.cc:1428] Setting key 5f318d04e7d7c517edc259f975b0b80b
Aug 11 17:22:08 dist-test-slave-mjjr krb5kdc[31253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468928, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.157.131@KRBTEST.COM; defaulting to no policy
I20260811 17:22:08.856211 31382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:08.841934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:57879 (local address 127.9.157.190:44901)
0811 17:22:08.842117 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:08.842121 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:08.843069 (+ 948us) server_negotiation.cc:407] Connection header received
0811 17:22:08.844721 (+ 1652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:08.844724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:08.844787 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:08.844900 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:08.846356 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:08.846912 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:08.848614 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:08.848861 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:08.849007 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:08.851915 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:08.851919 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:08.851921 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:08.851952 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:08.853479 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:08.854245 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:08.854248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:08.854250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:08.854345 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:08.854734 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:08.854737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:08.854739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:08.854963 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:08.855081 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:08.855766 (+ 685us) server_negotiation.cc:299] Negotiation successful
0811 17:22:08.856021 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 17:22:08.857298 31815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:08.842339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.130:57879)
0811 17:22:08.842857 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:08.842940 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:08.844480 (+ 1540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:08.845203 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:08.845211 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:08.845632 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:22:08.846189 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:08.846204 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:08.847064 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:08.847073 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:08.848356 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:08.848370 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:08.849067 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:08.849071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:08.849272 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:08.849940 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:22:08.849959 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:22:08.851638 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:22:08.853626 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:08.853633 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:08.853655 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:08.854096 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:08.854448 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:08.854452 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:08.854454 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:08.854621 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:08.855081 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:22:08.855091 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:22:08.855403 (+ 312us) client_negotiation.cc:769] Sending connection context
0811 17:22:08.855681 (+ 278us) client_negotiation.cc:240] Negotiation successful
0811 17:22:08.856235 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":133,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.131@KRBTEST.COM" created.
I20260811 17:22:08.858876 31811 heartbeater.cc:347] Connected to a master server at 127.9.157.190:44901
I20260811 17:22:08.859187 31811 heartbeater.cc:464] Registering TS with master...
I20260811 17:22:08.859930 31811 heartbeater.cc:511] Master 127.9.157.190:44901 requested a full tablet report, sending...
I20260811 17:22:08.861506 31320 ts_manager.cc:194] Registered new tserver with Master: 08aa841a1f38462ea7ab7c8fe54d9de0 (127.9.157.130:39529)
I20260811 17:22:08.862171 31320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.130@KRBTEST.COM'} at 127.9.157.130:57879
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
I20260811 17:22:08.878419 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:0
--local_ip_for_outbound_sockets=127.9.157.131
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:44901
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:22:09.007103 31822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:22:09.007545 31822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:22:09.007725 31822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:22:09.012168 31822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:22:09.012265 31822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:22:09.012332 31822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:22:09.018471 31822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:44901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:22:09.019793 31822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:22:09.021085 31822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:22:09.033464 31829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:09.033464 31833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:09.033509 31831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:09.034010 31822 server_base.cc:1034] running on GCE node
I20260811 17:22:09.034547 31822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:22:09.035341 31822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:22:09.036568 31822 hybrid_clock.cc:648] HybridClock initialized: now 1786468929036543 us; error 46 us; skew 500 ppm
Aug 11 17:22:09 dist-test-slave-mjjr krb5kdc[31253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468929, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:22:09.039904 31822 init.cc:377] Logged in from keytab as kudu/127.9.157.131@KRBTEST.COM (short username kudu)
I20260811 17:22:09.040946 31822 webserver.cc:460] Webserver started at http://127.9.157.131:35451/ using document root <none> and password file <none>
I20260811 17:22:09.041556 31822 fs_manager.cc:359] Metadata directory not provided
I20260811 17:22:09.041625 31822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:22:09.041831 31822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:22:09.044234 31822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/data/instance:
uuid: "ceb96fcae8fc40b7a0d2f73592090af9"
format_stamp: "Formatted at 2026-08-11 17:22:09 on dist-test-slave-mjjr"
server_key: "7a0bb02280328a7f2dda16ac32d699f7"
server_key_iv: "e7b4e0cb44cb1647d22c73961f1fc1e1"
server_key_version: "encryptionkey@0"
I20260811 17:22:09.044866 31822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance:
uuid: "ceb96fcae8fc40b7a0d2f73592090af9"
format_stamp: "Formatted at 2026-08-11 17:22:09 on dist-test-slave-mjjr"
server_key: "7a0bb02280328a7f2dda16ac32d699f7"
server_key_iv: "e7b4e0cb44cb1647d22c73961f1fc1e1"
server_key_version: "encryptionkey@0"
I20260811 17:22:09.050077 31822 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 17:22:09.053624 31844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:09.056185 31822 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 17:22:09.056428 31822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "ceb96fcae8fc40b7a0d2f73592090af9"
format_stamp: "Formatted at 2026-08-11 17:22:09 on dist-test-slave-mjjr"
server_key: "7a0bb02280328a7f2dda16ac32d699f7"
server_key_iv: "e7b4e0cb44cb1647d22c73961f1fc1e1"
server_key_version: "encryptionkey@0"
I20260811 17:22:09.056620 31822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:22:09.074201 31822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:22:09.077863 31822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:22:09.078073 31822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:22:09.078867 31822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:22:09.079955 31822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:22:09.080013 31822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:09.080084 31822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:22:09.080117 31822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:09.091708 31822 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:40565
I20260811 17:22:09.091918 32000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:40565 every 8 connection(s)
I20260811 17:22:09.092849 31822 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:22:09.098150 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 31822
I20260811 17:22:09.098356 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance
I20260811 17:22:09.098657 9846 external_mini_cluster.cc:1428] Setting key 50219a08aa18a05507f03c8618fcb3dd
Aug 11 17:22:09 dist-test-slave-mjjr krb5kdc[31253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468929, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:22:09.109759 31382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:09.095047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:54633 (local address 127.9.157.190:44901)
0811 17:22:09.095199 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:09.095203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:09.096167 (+ 964us) server_negotiation.cc:407] Connection header received
0811 17:22:09.097605 (+ 1438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:09.097609 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:09.097677 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:09.097795 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:09.099465 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.100228 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.101585 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.101820 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.101935 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:09.105272 (+ 3337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:09.105276 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:09.105278 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:09.105308 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:09.107232 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:09.107978 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:09.107983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:09.107990 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:09.108074 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:09.108455 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:09.108459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:09.108460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:09.108675 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:09.108784 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:09.109394 (+ 610us) server_negotiation.cc:299] Negotiation successful
0811 17:22:09.109543 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 17:22:09.110673 32003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:09.095394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.131:54633)
0811 17:22:09.095993 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:09.096045 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:09.097362 (+ 1317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:09.098010 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:09.098018 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:09.098561 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:22:09.099305 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:09.099326 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.100366 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.100370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:09.101441 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:09.101453 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.101997 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.102001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:09.102196 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:09.102865 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:22:09.102905 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:22:09.105030 (+ 2125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:22:09.107421 (+ 2391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:09.107435 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:09.107460 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:09.107862 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:09.108176 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:09.108180 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:09.108182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:09.108343 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:09.108802 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:22:09.108810 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:22:09.109113 (+ 303us) client_negotiation.cc:769] Sending connection context
0811 17:22:09.109322 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 17:22:09.109705 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":163,"threads_started":1}
I20260811 17:22:09.112365 32001 heartbeater.cc:347] Connected to a master server at 127.9.157.190:44901
I20260811 17:22:09.112656 32001 heartbeater.cc:464] Registering TS with master...
I20260811 17:22:09.113247 32001 heartbeater.cc:511] Master 127.9.157.190:44901 requested a full tablet report, sending...
I20260811 17:22:09.114403 31320 ts_manager.cc:194] Registered new tserver with Master: ceb96fcae8fc40b7a0d2f73592090af9 (127.9.157.131:40565)
I20260811 17:22:09.115005 31320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.131@KRBTEST.COM'} at 127.9.157.131:54633
I20260811 17:22:09.125876 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:22:09.136689 31382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:09.128593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42094 (local address 127.9.157.190:44901)
0811 17:22:09.128734 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:09.128738 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:09.128823 (+ 85us) server_negotiation.cc:407] Connection header received
0811 17:22:09.128936 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:09.128939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:09.128985 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:09.129066 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:09.129997 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.130535 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.131414 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.131672 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.131769 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:09.133381 (+ 1612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:09.133385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:09.133387 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:09.133415 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:09.134812 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:09.135250 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:09.135253 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:09.135254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:09.135342 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:09.135708 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:09.135714 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:09.135719 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:09.136021 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:09.136146 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:09.136432 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 17:22:09.136552 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 17:22:09.140359 31320 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42094:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:22:09.143811 31320 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:22:09.160238 32013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:09.154042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:39529 (local address 127.0.0.1:44642)
0811 17:22:09.154840 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:09.154901 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:09.155135 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:09.155950 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:09.155956 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:09.155982 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:22:09.156121 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:09.156127 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.157347 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.157354 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:09.158683 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:09.158694 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.159592 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.159595 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:09.159742 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:09.159786 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 17:22:09.159929 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 17:22:09.160071 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"mutex_wait_us":32,"thread_start_us":111,"threads_started":1}
I20260811 17:22:09.160251 32014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:09.154186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44642 (local address 127.9.157.130:39529)
0811 17:22:09.154685 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:09.154698 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:09.154999 (+ 301us) server_negotiation.cc:407] Connection header received
0811 17:22:09.155355 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:09.155362 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:09.155630 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:09.155828 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:22:09.156256 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.157203 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.158828 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.159447 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.159584 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:09.159795 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:09.159954 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 17:22:09.160069 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":159,"threads_started":1}
I20260811 17:22:09.161707 32015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:09.154046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:32955 (local address 127.0.0.1:42896)
0811 17:22:09.155093 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:09.155107 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:09.155538 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:09.156268 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:09.156271 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:09.156287 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:22:09.156411 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:09.156422 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.157905 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.157911 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:09.159626 (+ 1715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:09.159672 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.161122 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.161126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:09.161290 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:09.161352 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 17:22:09.161423 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 17:22:09.161515 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":930,"mutex_wait_us":476,"thread_start_us":117,"threads_started":1}
I20260811 17:22:09.161761 32016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:09.154289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42896 (local address 127.9.157.129:32955)
0811 17:22:09.154908 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:09.154914 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:09.155214 (+ 300us) server_negotiation.cc:407] Connection header received
0811 17:22:09.155743 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:09.155750 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:09.155950 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:09.156069 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:22:09.156551 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.157753 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.159820 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.160829 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.160904 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:09.161032 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:09.161498 (+ 466us) server_negotiation.cc:299] Negotiation successful
0811 17:22:09.161614 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":161,"threads_started":1}
I20260811 17:22:09.162190 32019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:09.155392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53374 (local address 127.9.157.131:40565)
0811 17:22:09.155787 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:09.155799 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:09.156065 (+ 266us) server_negotiation.cc:407] Connection header received
0811 17:22:09.156144 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:09.156149 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:09.156325 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:09.156481 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:22:09.156724 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.157828 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.159820 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.160683 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.160875 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:09.161032 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:09.161790 (+ 758us) server_negotiation.cc:299] Negotiation successful
0811 17:22:09.162037 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":115,"threads_started":1}
I20260811 17:22:09.161713 32017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:09.154905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:40565 (local address 127.0.0.1:53374)
0811 17:22:09.155838 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:09.155851 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:09.155962 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:09.156480 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:09.156482 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:09.156503 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:22:09.156590 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:09.156597 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.157968 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.157972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:09.159666 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:09.159672 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.161106 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.161110 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:09.161277 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:09.161350 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 17:22:09.161423 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 17:22:09.161515 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":870,"mutex_wait_us":313,"thread_start_us":101,"threads_started":1}
I20260811 17:22:09.164047 31717 tablet_service.cc:1467] Processing CreateTablet for tablet 8b9c47e6f3994a1bb30641b3ffe1a02b (DEFAULT_TABLE table=test-table [id=d7a1027a35014c45b6733ce52cb649a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:22:09.165578 31717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b9c47e6f3994a1bb30641b3ffe1a02b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:22:09.165171 31513 tablet_service.cc:1467] Processing CreateTablet for tablet 8b9c47e6f3994a1bb30641b3ffe1a02b (DEFAULT_TABLE table=test-table [id=d7a1027a35014c45b6733ce52cb649a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:22:09.165550 31913 tablet_service.cc:1467] Processing CreateTablet for tablet 8b9c47e6f3994a1bb30641b3ffe1a02b (DEFAULT_TABLE table=test-table [id=d7a1027a35014c45b6733ce52cb649a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:22:09.166290 31513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b9c47e6f3994a1bb30641b3ffe1a02b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:22:09.166599 31913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b9c47e6f3994a1bb30641b3ffe1a02b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:22:09.174916 32025 tablet_bootstrap.cc:492] T 8b9c47e6f3994a1bb30641b3ffe1a02b P ceb96fcae8fc40b7a0d2f73592090af9: Bootstrap starting.
I20260811 17:22:09.175078 32024 tablet_bootstrap.cc:492] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 08aa841a1f38462ea7ab7c8fe54d9de0: Bootstrap starting.
I20260811 17:22:09.177516 32025 tablet_bootstrap.cc:654] T 8b9c47e6f3994a1bb30641b3ffe1a02b P ceb96fcae8fc40b7a0d2f73592090af9: Neither blocks nor log segments found. Creating new log.
I20260811 17:22:09.178138 32024 tablet_bootstrap.cc:654] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 08aa841a1f38462ea7ab7c8fe54d9de0: Neither blocks nor log segments found. Creating new log.
I20260811 17:22:09.178802 32025 log.cc:824] T 8b9c47e6f3994a1bb30641b3ffe1a02b P ceb96fcae8fc40b7a0d2f73592090af9: Log is configured to *not* fsync() on all Append() calls
I20260811 17:22:09.178964 32024 log.cc:824] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 08aa841a1f38462ea7ab7c8fe54d9de0: Log is configured to *not* fsync() on all Append() calls
I20260811 17:22:09.179941 32023 tablet_bootstrap.cc:492] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3: Bootstrap starting.
I20260811 17:22:09.181406 32024 tablet_bootstrap.cc:492] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 08aa841a1f38462ea7ab7c8fe54d9de0: No bootstrap required, opened a new log
I20260811 17:22:09.181507 32025 tablet_bootstrap.cc:492] T 8b9c47e6f3994a1bb30641b3ffe1a02b P ceb96fcae8fc40b7a0d2f73592090af9: No bootstrap required, opened a new log
I20260811 17:22:09.181697 32024 ts_tablet_manager.cc:1397] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 08aa841a1f38462ea7ab7c8fe54d9de0: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 17:22:09.181734 32025 ts_tablet_manager.cc:1397] T 8b9c47e6f3994a1bb30641b3ffe1a02b P ceb96fcae8fc40b7a0d2f73592090af9: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 17:22:09.183109 32023 tablet_bootstrap.cc:654] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3: Neither blocks nor log segments found. Creating new log.
I20260811 17:22:09.184181 32023 log.cc:824] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3: Log is configured to *not* fsync() on all Append() calls
I20260811 17:22:09.185199 32024 raft_consensus.cc:348] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 08aa841a1f38462ea7ab7c8fe54d9de0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } }
I20260811 17:22:09.185611 32024 raft_consensus.cc:374] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 08aa841a1f38462ea7ab7c8fe54d9de0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:22:09.185732 32024 raft_consensus.cc:729] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 08aa841a1f38462ea7ab7c8fe54d9de0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08aa841a1f38462ea7ab7c8fe54d9de0, State: Initialized, Role: FOLLOWER
I20260811 17:22:09.185879 32025 raft_consensus.cc:348] T 8b9c47e6f3994a1bb30641b3ffe1a02b P ceb96fcae8fc40b7a0d2f73592090af9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } }
I20260811 17:22:09.186178 32025 raft_consensus.cc:374] T 8b9c47e6f3994a1bb30641b3ffe1a02b P ceb96fcae8fc40b7a0d2f73592090af9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:22:09.186268 32025 raft_consensus.cc:729] T 8b9c47e6f3994a1bb30641b3ffe1a02b P ceb96fcae8fc40b7a0d2f73592090af9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb96fcae8fc40b7a0d2f73592090af9, State: Initialized, Role: FOLLOWER
I20260811 17:22:09.186393 32024 consensus_queue.cc:260] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 08aa841a1f38462ea7ab7c8fe54d9de0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } }
I20260811 17:22:09.186748 32025 consensus_queue.cc:260] T 8b9c47e6f3994a1bb30641b3ffe1a02b P ceb96fcae8fc40b7a0d2f73592090af9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } }
I20260811 17:22:09.187172 32023 tablet_bootstrap.cc:492] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3: No bootstrap required, opened a new log
I20260811 17:22:09.187398 32023 ts_tablet_manager.cc:1397] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 17:22:09.187697 32024 ts_tablet_manager.cc:1428] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 08aa841a1f38462ea7ab7c8fe54d9de0: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260811 17:22:09.187824 32025 ts_tablet_manager.cc:1428] T 8b9c47e6f3994a1bb30641b3ffe1a02b P ceb96fcae8fc40b7a0d2f73592090af9: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260811 17:22:09.188253 32001 heartbeater.cc:503] Master 127.9.157.190:44901 was elected leader, sending a full tablet report...
I20260811 17:22:09.188232 31811 heartbeater.cc:503] Master 127.9.157.190:44901 was elected leader, sending a full tablet report...
I20260811 17:22:09.195793 32023 raft_consensus.cc:348] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } }
I20260811 17:22:09.196183 32023 raft_consensus.cc:374] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:22:09.196285 32023 raft_consensus.cc:729] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6308b38e7b554a12861b5e97ab5c0aa3, State: Initialized, Role: FOLLOWER
I20260811 17:22:09.196847 32023 consensus_queue.cc:260] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } }
I20260811 17:22:09.197734 31612 heartbeater.cc:503] Master 127.9.157.190:44901 was elected leader, sending a full tablet report...
I20260811 17:22:09.198135 32023 ts_tablet_manager.cc:1428] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
W20260811 17:22:09.312646 31613 tablet.cc:2369] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:22:09.341259 31813 tablet.cc:2369] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 08aa841a1f38462ea7ab7c8fe54d9de0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:22:09.343990 32002 tablet.cc:2369] T 8b9c47e6f3994a1bb30641b3ffe1a02b P ceb96fcae8fc40b7a0d2f73592090af9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:22:09.372026 32038 raft_consensus.cc:482] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:22:09.372308 32038 raft_consensus.cc:504] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } }
I20260811 17:22:09.373590 32038 leader_election.cc:288] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08aa841a1f38462ea7ab7c8fe54d9de0 (127.9.157.130:39529), ceb96fcae8fc40b7a0d2f73592090af9 (127.9.157.131:40565)
I20260811 17:22:09.378394 32014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:09.374079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:58781 (local address 127.9.157.130:39529)
0811 17:22:09.374225 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:09.374229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:09.374773 (+ 544us) server_negotiation.cc:407] Connection header received
0811 17:22:09.374850 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:09.374854 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:09.374914 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:09.375006 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:22:09.375437 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.376118 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.377334 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.377756 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.377900 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:09.377967 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:09.378154 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 17:22:09.378255 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 17:22:09.378409 32040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:09.374001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:39529 (local address 127.9.157.129:58781)
0811 17:22:09.374390 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:09.374408 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:09.374542 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:09.375050 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:09.375053 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:09.375100 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:22:09.375278 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:09.375287 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.376235 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.376239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:09.377191 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:09.377202 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.377900 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.377904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:09.378005 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:09.378035 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 17:22:09.378108 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 17:22:09.378202 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":1,"thread_start_us":188,"threads_started":1}
I20260811 17:22:09.378459 32019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:09.374733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:50569 (local address 127.9.157.131:40565)
0811 17:22:09.374907 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:09.374911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:09.375107 (+ 196us) server_negotiation.cc:407] Connection header received
0811 17:22:09.375234 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:09.375238 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:09.375290 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:09.375386 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:22:09.375744 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.376274 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.377368 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.377788 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.377935 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:09.377996 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:09.378238 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 17:22:09.378325 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 17:22:09.378417 32041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:09.374547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:40565 (local address 127.9.157.129:50569)
0811 17:22:09.375006 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:09.375021 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:09.375124 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:09.375444 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:09.375447 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:09.375468 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:22:09.375599 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:09.375606 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.376396 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.376399 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:09.377208 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:09.377218 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.377956 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.377960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:09.378072 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:09.378118 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 17:22:09.378201 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 17:22:09.378269 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"mutex_wait_us":24,"thread_start_us":108,"threads_started":1}
I20260811 17:22:09.379442 31941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b9c47e6f3994a1bb30641b3ffe1a02b" candidate_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" is_pre_election: true
I20260811 17:22:09.379444 31750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b9c47e6f3994a1bb30641b3ffe1a02b" candidate_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" is_pre_election: true
I20260811 17:22:09.379827 31750 raft_consensus.cc:2454] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 08aa841a1f38462ea7ab7c8fe54d9de0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6308b38e7b554a12861b5e97ab5c0aa3 in term 0.
I20260811 17:22:09.379840 31941 raft_consensus.cc:2454] T 8b9c47e6f3994a1bb30641b3ffe1a02b P ceb96fcae8fc40b7a0d2f73592090af9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6308b38e7b554a12861b5e97ab5c0aa3 in term 0.
I20260811 17:22:09.380398 31442 leader_election.cc:302] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6308b38e7b554a12861b5e97ab5c0aa3, ceb96fcae8fc40b7a0d2f73592090af9; no voters:
I20260811 17:22:09.380743 32038 raft_consensus.cc:2781] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:22:09.380831 32038 raft_consensus.cc:482] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:22:09.380862 32038 raft_consensus.cc:3037] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:22:09.382055 32038 raft_consensus.cc:504] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } }
I20260811 17:22:09.382474 32038 leader_election.cc:288] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3 [CANDIDATE]: Term 1 election: Requested vote from peers 08aa841a1f38462ea7ab7c8fe54d9de0 (127.9.157.130:39529), ceb96fcae8fc40b7a0d2f73592090af9 (127.9.157.131:40565)
I20260811 17:22:09.382985 31750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b9c47e6f3994a1bb30641b3ffe1a02b" candidate_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0"
I20260811 17:22:09.383059 31941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b9c47e6f3994a1bb30641b3ffe1a02b" candidate_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb96fcae8fc40b7a0d2f73592090af9"
I20260811 17:22:09.383116 31750 raft_consensus.cc:3037] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 08aa841a1f38462ea7ab7c8fe54d9de0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:22:09.383235 31941 raft_consensus.cc:3037] T 8b9c47e6f3994a1bb30641b3ffe1a02b P ceb96fcae8fc40b7a0d2f73592090af9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:22:09.384322 31750 raft_consensus.cc:2454] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 08aa841a1f38462ea7ab7c8fe54d9de0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6308b38e7b554a12861b5e97ab5c0aa3 in term 1.
I20260811 17:22:09.384476 31941 raft_consensus.cc:2454] T 8b9c47e6f3994a1bb30641b3ffe1a02b P ceb96fcae8fc40b7a0d2f73592090af9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6308b38e7b554a12861b5e97ab5c0aa3 in term 1.
I20260811 17:22:09.384724 31442 leader_election.cc:302] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08aa841a1f38462ea7ab7c8fe54d9de0, 6308b38e7b554a12861b5e97ab5c0aa3; no voters:
I20260811 17:22:09.384931 32038 raft_consensus.cc:2781] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:22:09.385169 32038 raft_consensus.cc:686] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3 [term 1 LEADER]: Becoming Leader. State: Replica: 6308b38e7b554a12861b5e97ab5c0aa3, State: Running, Role: LEADER
I20260811 17:22:09.385655 32038 consensus_queue.cc:237] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } }
I20260811 17:22:09.389982 31320 catalog_manager.cc:5495] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6308b38e7b554a12861b5e97ab5c0aa3 (127.9.157.129). New cstate: current_term: 1 leader_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } health_report { overall_health: HEALTHY } } }
I20260811 17:22:09.405073 32016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:09.401396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42908 (local address 127.9.157.129:32955)
0811 17:22:09.401550 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:09.401554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:09.401567 (+ 13us) server_negotiation.cc:407] Connection header received
0811 17:22:09.401657 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:09.401659 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:09.401716 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:09.401815 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:22:09.402039 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.402551 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.403400 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.403619 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.403771 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:09.404113 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:22:09.404609 (+ 496us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:22:09.404716 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:09.404811 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 17:22:09.404872 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 17:22:09.412117 31750 raft_consensus.cc:1261] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 08aa841a1f38462ea7ab7c8fe54d9de0 [term 1 FOLLOWER]: Refusing update from remote peer 6308b38e7b554a12861b5e97ab5c0aa3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:22:09.412117 31941 raft_consensus.cc:1261] T 8b9c47e6f3994a1bb30641b3ffe1a02b P ceb96fcae8fc40b7a0d2f73592090af9 [term 1 FOLLOWER]: Refusing update from remote peer 6308b38e7b554a12861b5e97ab5c0aa3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:22:09.413265 32042 consensus_queue.cc:1035] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:22:09.413553 32038 consensus_queue.cc:1035] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:22:09.421221 32046 mvcc.cc:205] Tried to move back new op lower bound from 7317376734862721024 to 7317376734769520640. Current Snapshot: MvccSnapshot[applied={T|T < 7317376734862721024}]
I20260811 17:22:09.433897 31319 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:22:09.434084 31319 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:22:09.437078 31319 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0: Sending DeleteTablet for 3 replicas of tablet 8b9c47e6f3994a1bb30641b3ffe1a02b
I20260811 17:22:09.438179 31513 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b9c47e6f3994a1bb30641b3ffe1a02b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:22:09 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:42896
I20260811 17:22:09.438195 31717 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b9c47e6f3994a1bb30641b3ffe1a02b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:22:09 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:44642
I20260811 17:22:09.438195 31913 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b9c47e6f3994a1bb30641b3ffe1a02b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:22:09 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:53374
I20260811 17:22:09.438841 32060 tablet_replica.cc:330] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3: stopping tablet replica
I20260811 17:22:09.438911 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 31417
I20260811 17:22:09.439334 32060 raft_consensus.cc:2229] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3 [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:22:09.439724 32058 tablet_replica.cc:330] T 8b9c47e6f3994a1bb30641b3ffe1a02b P ceb96fcae8fc40b7a0d2f73592090af9: stopping tablet replica
I20260811 17:22:09.440141 32058 raft_consensus.cc:2229] T 8b9c47e6f3994a1bb30641b3ffe1a02b P ceb96fcae8fc40b7a0d2f73592090af9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:22:09.440615 32058 raft_consensus.cc:2258] T 8b9c47e6f3994a1bb30641b3ffe1a02b P ceb96fcae8fc40b7a0d2f73592090af9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:22:09.445931 32059 tablet_replica.cc:330] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 08aa841a1f38462ea7ab7c8fe54d9de0: stopping tablet replica
I20260811 17:22:09.446386 32059 raft_consensus.cc:2229] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 08aa841a1f38462ea7ab7c8fe54d9de0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:22:09.446810 32059 raft_consensus.cc:2258] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 08aa841a1f38462ea7ab7c8fe54d9de0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:22:09.448331 32058 ts_tablet_manager.cc:1905] T 8b9c47e6f3994a1bb30641b3ffe1a02b P ceb96fcae8fc40b7a0d2f73592090af9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:22:09.452669 32058 ts_tablet_manager.cc:1918] T 8b9c47e6f3994a1bb30641b3ffe1a02b P ceb96fcae8fc40b7a0d2f73592090af9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:22:09.452862 32058 log.cc:1196] T 8b9c47e6f3994a1bb30641b3ffe1a02b P ceb96fcae8fc40b7a0d2f73592090af9: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/wal/wals/8b9c47e6f3994a1bb30641b3ffe1a02b
I20260811 17:22:09.453204 32058 ts_tablet_manager.cc:1939] T 8b9c47e6f3994a1bb30641b3ffe1a02b P ceb96fcae8fc40b7a0d2f73592090af9: Deleting consensus metadata
I20260811 17:22:09.453742 32059 ts_tablet_manager.cc:1905] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 08aa841a1f38462ea7ab7c8fe54d9de0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:22:09.454756 31300 catalog_manager.cc:4841] TS ceb96fcae8fc40b7a0d2f73592090af9 (127.9.157.131:40565): tablet 8b9c47e6f3994a1bb30641b3ffe1a02b (table test-table [id=d7a1027a35014c45b6733ce52cb649a5]) successfully deleted
I20260811 17:22:09.457790 32059 ts_tablet_manager.cc:1918] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 08aa841a1f38462ea7ab7c8fe54d9de0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:22:09.457950 32059 log.cc:1196] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 08aa841a1f38462ea7ab7c8fe54d9de0: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/wal/wals/8b9c47e6f3994a1bb30641b3ffe1a02b
I20260811 17:22:09.458315 32059 ts_tablet_manager.cc:1939] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 08aa841a1f38462ea7ab7c8fe54d9de0: Deleting consensus metadata
I20260811 17:22:09.460096 31300 catalog_manager.cc:4841] TS 08aa841a1f38462ea7ab7c8fe54d9de0 (127.9.157.130:39529): tablet 8b9c47e6f3994a1bb30641b3ffe1a02b (table test-table [id=d7a1027a35014c45b6733ce52cb649a5]) successfully deleted
W20260811 17:22:09.461051 31301 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.157.129:32955 (error 108)
I20260811 17:22:09.461369 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 31621
I20260811 17:22:09.465054 32017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:09.464589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:32955 (local address 127.0.0.1:42912)
0811 17:22:09.464799 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:09.464917 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.129:32955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 17:22:09.465369 31301 catalog_manager.cc:4584] TS 6308b38e7b554a12861b5e97ab5c0aa3 (127.9.157.129:32955): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8b9c47e6f3994a1bb30641b3ffe1a02b: Network error: Client connection negotiation failed: client connection to 127.9.157.129:32955: connect: Connection refused (error 111)
I20260811 17:22:09.474977 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 31822
I20260811 17:22:09.492192 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 31272
I20260811 17:22:09.515734 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:44901
--webserver_interface=127.9.157.190
--webserver_port=37181
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:44901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:22:09.641396 32063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:22:09.641726 32063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:22:09.641773 32063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:22:09.645969 32063 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:22:09.646060 32063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:22:09.646080 32063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:22:09.646096 32063 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:22:09.646113 32063 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:22:09.652266 32063 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:44901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:44901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=37181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:22:09.653542 32063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:22:09.654654 32063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:22:09.671625 32079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:09.671613 32075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:09.671613 32073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:09.672401 32063 server_base.cc:1034] running on GCE node
I20260811 17:22:09.673209 32063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:22:09.674695 32063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:22:09.675884 32063 hybrid_clock.cc:648] HybridClock initialized: now 1786468929675860 us; error 38 us; skew 500 ppm
Aug 11 17:22:09 dist-test-slave-mjjr krb5kdc[31253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468929, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:22:09.679504 32063 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:22:09.681154 32063 webserver.cc:460] Webserver started at http://127.9.157.190:37181/ using document root <none> and password file <none>
I20260811 17:22:09.682325 32063 fs_manager.cc:359] Metadata directory not provided
I20260811 17:22:09.682512 32063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:22:09.689137 32063 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 17:22:09.692432 32087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:09.694279 32063 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:22:09.694581 32063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0"
format_stamp: "Formatted at 2026-08-11 17:22:08 on dist-test-slave-mjjr"
server_key: "b1e218ce3e3d12af42b137bf90cf6fc5"
server_key_iv: "89e7f2fe7ff6fcfcdd4e86584a1b38a6"
server_key_version: "encryptionkey@0"
I20260811 17:22:09.695345 32063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:22:09.711743 32063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:22:09.715521 32063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:22:09.715828 32063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:22:09.724273 32063 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:44901
I20260811 17:22:09.724619 32159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:44901 every 8 connection(s)
I20260811 17:22:09.725565 32063 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:22:09.730696 32161 sys_catalog.cc:263] Verifying existing consensus state
I20260811 17:22:09.732714 32161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0: Bootstrap starting.
I20260811 17:22:09.735389 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 32063
I20260811 17:22:09.735960 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:32955
--local_ip_for_outbound_sockets=127.9.157.129
--tserver_master_addrs=127.9.157.190:44901
--webserver_port=33223
--webserver_interface=127.9.157.129
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:22:09.752918 32161 log.cc:824] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0: Log is configured to *not* fsync() on all Append() calls
I20260811 17:22:09.762714 32161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 17:22:09.763195 32161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0: Bootstrap complete.
I20260811 17:22:09.768796 32161 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44901 } }
I20260811 17:22:09.769236 32161 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab1d7cfe1fe9459b9e20d6139e27a4b0, State: Initialized, Role: FOLLOWER
I20260811 17:22:09.769825 32161 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44901 } }
I20260811 17:22:09.769951 32161 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:22:09.770045 32161 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:22:09.770165 32161 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [term 1 FOLLOWER]: Advancing to term 2
I20260811 17:22:09.772475 32161 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44901 } }
I20260811 17:22:09.772873 32161 leader_election.cc:302] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab1d7cfe1fe9459b9e20d6139e27a4b0; no voters:
I20260811 17:22:09.773277 32161 leader_election.cc:288] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 17:22:09.773386 32169 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 17:22:09.773586 32169 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [term 2 LEADER]: Becoming Leader. State: Replica: ab1d7cfe1fe9459b9e20d6139e27a4b0, State: Running, Role: LEADER
I20260811 17:22:09.773916 32169 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44901 } }
I20260811 17:22:09.774729 32161 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:22:09.775003 32171 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab1d7cfe1fe9459b9e20d6139e27a4b0. Latest consensus state: current_term: 2 leader_uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44901 } } }
I20260811 17:22:09.775133 32171 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [sys.catalog]: This master's current role is: LEADER
I20260811 17:22:09.775441 32170 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44901 } } }
I20260811 17:22:09.775545 32170 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [sys.catalog]: This master's current role is: LEADER
I20260811 17:22:09.775552 32176 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:22:09.782214 32176 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:22:09.783066 32176 catalog_manager.cc:1259] Loaded cluster ID: 74fa366ef4d14041a98b9ee9435ddc01
I20260811 17:22:09.783152 32176 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:22:09.785121 32176 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 17:22:09.789738 32176 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 17:22:09.796578 32176 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:22:09.797899 32176 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0: Loaded TSK: 0
I20260811 17:22:09.799317 32176 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 17:22:09.877420 32165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:22:09.877751 32165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:22:09.877836 32165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:22:09.883567 32165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:22:09.883698 32165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:22:09.883800 32165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:22:09.890741 32165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:32955
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=33223
--tserver_master_addrs=127.9.157.190:44901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:22:09.891990 32165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:22:09.893126 32165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:22:09.905340 32202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:09.905485 32205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:09.905576 32165 server_base.cc:1034] running on GCE node
W20260811 17:22:09.905332 32203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:09.906261 32165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:22:09.907073 32165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:22:09.908262 32165 hybrid_clock.cc:648] HybridClock initialized: now 1786468929908232 us; error 55 us; skew 500 ppm
Aug 11 17:22:09 dist-test-slave-mjjr krb5kdc[31253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468929, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:22:09.911494 32165 init.cc:377] Logged in from keytab as kudu/127.9.157.129@KRBTEST.COM (short username kudu)
I20260811 17:22:09.912671 32165 webserver.cc:460] Webserver started at http://127.9.157.129:33223/ using document root <none> and password file <none>
I20260811 17:22:09.913336 32165 fs_manager.cc:359] Metadata directory not provided
I20260811 17:22:09.913429 32165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:22:09.918135 32165 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 17:22:09.920670 32215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:09.922145 32165 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:22:09.922299 32165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "6308b38e7b554a12861b5e97ab5c0aa3"
format_stamp: "Formatted at 2026-08-11 17:22:08 on dist-test-slave-mjjr"
server_key: "714c6f8b86c4ddec405bdc59cbdb79aa"
server_key_iv: "d9bfd631d23c0920869b56b086a46857"
server_key_version: "encryptionkey@0"
I20260811 17:22:09.922967 32165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:22:09.947856 32165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:22:09.951596 32165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:22:09.951890 32165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:22:09.952610 32165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:22:09.953930 32225 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 17:22:09.959022 32165 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 17:22:09.959127 32165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 17:22:09.959237 32165 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 17:22:09.961283 32165 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 17:22:09.961344 32165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260811 17:22:09.961691 32225 tablet_bootstrap.cc:492] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3: Bootstrap starting.
I20260811 17:22:09.973982 32165 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:32955
I20260811 17:22:09.974051 32399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:32955 every 8 connection(s)
I20260811 17:22:09.975145 32165 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:22:09.977874 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 32165
I20260811 17:22:09.978399 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:39529
--local_ip_for_outbound_sockets=127.9.157.130
--tserver_master_addrs=127.9.157.190:44901
--webserver_port=45189
--webserver_interface=127.9.157.130
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:22:09.982709 32225 log.cc:824] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3: Log is configured to *not* fsync() on all Append() calls
Aug 11 17:22:09 dist-test-slave-mjjr krb5kdc[31253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468929, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:22:09.991873 32225 tablet_bootstrap.cc:492] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 17:22:09.992412 32225 tablet_bootstrap.cc:492] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3: Bootstrap complete.
I20260811 17:22:09.993867 32225 ts_tablet_manager.cc:1397] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3: Time spent bootstrapping tablet: real 0.032s user 0.026s sys 0.004s
I20260811 17:22:09.997021 32225 raft_consensus.cc:348] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } }
I20260811 17:22:09.997613 32225 raft_consensus.cc:729] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6308b38e7b554a12861b5e97ab5c0aa3, State: Initialized, Role: FOLLOWER
I20260811 17:22:09.998188 32225 consensus_queue.cc:260] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } }
I20260811 17:22:09.998680 32404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:09.979302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:48497 (local address 127.9.157.190:44901)
0811 17:22:09.979909 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:09.979931 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:09.979985 (+ 54us) server_negotiation.cc:407] Connection header received
0811 17:22:09.980982 (+ 997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:09.981001 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:09.981482 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:09.981902 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:09.983464 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.984965 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.987221 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.987987 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.988178 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:09.991750 (+ 3572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:09.991760 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:09.991782 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:09.991833 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:09.994572 (+ 2739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:09.995293 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:09.995300 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:09.995306 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:09.995491 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:09.995940 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:09.995944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:09.995946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:09.996468 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:09.996605 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:09.997304 (+ 699us) server_negotiation.cc:299] Negotiation successful
0811 17:22:09.997649 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":137,"threads_started":1}
I20260811 17:22:09.999068 32403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:09.977989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.129:48497)
0811 17:22:09.978974 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:09.979048 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:09.980417 (+ 1369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:09.982001 (+ 1584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:09.982010 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:09.982528 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:22:09.983250 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:09.983273 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.985187 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.985194 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:09.987019 (+ 1825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:09.987039 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:09.988179 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:09.988186 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:09.988439 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:09.989229 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:22:09.989259 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:22:09.991423 (+ 2164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:22:09.994744 (+ 3321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:09.994751 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:09.994768 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:09.995140 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:09.995614 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:09.995618 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:09.995620 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:09.995828 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:09.996627 (+ 799us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:22:09.996635 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:22:09.996944 (+ 309us) client_negotiation.cc:769] Sending connection context
0811 17:22:09.997162 (+ 218us) client_negotiation.cc:240] Negotiation successful
0811 17:22:09.997550 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":194,"threads_started":1}
I20260811 17:22:09.999351 32225 ts_tablet_manager.cc:1428] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260811 17:22:10.001880 32401 heartbeater.cc:347] Connected to a master server at 127.9.157.190:44901
I20260811 17:22:10.002173 32401 heartbeater.cc:464] Registering TS with master...
I20260811 17:22:10.002743 32401 heartbeater.cc:511] Master 127.9.157.190:44901 requested a full tablet report, sending...
I20260811 17:22:10.005587 32111 ts_manager.cc:194] Registered new tserver with Master: 6308b38e7b554a12861b5e97ab5c0aa3 (127.9.157.129:32955)
I20260811 17:22:10.009851 32111 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.129@KRBTEST.COM'} at 127.9.157.129:48497
I20260811 17:22:10.012782 32401 heartbeater.cc:503] Master 127.9.157.190:44901 was elected leader, sending a full tablet report...
Aug 11 17:22:10 dist-test-slave-mjjr krb5kdc[31253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468929, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for kudu/127.9.157.129@KRBTEST.COM
I20260811 17:22:10.022142 32411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:10.009394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:32955 (local address 127.0.0.1:42918)
0811 17:22:10.010309 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:10.010356 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:10.010520 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:10.011734 (+ 1214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:10.011744 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:10.012234 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:22:10.012378 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:10.012388 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.013801 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.013805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:10.014757 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:10.014769 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.015378 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.015381 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:10.015499 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:10.016204 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:22:10.016217 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:22:10.017917 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:22:10.020164 (+ 2247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:10.020175 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:10.020182 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:10.020449 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:10.020785 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:10.020788 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:10.020790 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:10.020885 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:10.021390 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:22:10.021397 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:22:10.021539 (+ 142us) client_negotiation.cc:769] Sending connection context
0811 17:22:10.021688 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 17:22:10.021933 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"thread_start_us":137,"threads_started":1}
I20260811 17:22:10.022145 32412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:10.009525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42918 (local address 127.9.157.129:32955)
0811 17:22:10.010019 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:10.010033 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:10.010486 (+ 453us) server_negotiation.cc:407] Connection header received
0811 17:22:10.010690 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:10.010695 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:10.010914 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:10.011063 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:10.012544 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.013664 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.014881 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.015241 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.015371 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:10.018099 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:10.018110 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:10.018122 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:10.018150 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:10.020028 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:10.020590 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:10.020595 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:10.020597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:10.020682 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:10.020981 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:10.020985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:10.020987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:10.021270 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:10.021419 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:10.021797 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 17:22:10.021996 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":120,"threads_started":1}
I20260811 17:22:10.023443 32289 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b9c47e6f3994a1bb30641b3ffe1a02b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:22:09 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:42918
I20260811 17:22:10.024643 32413 tablet_replica.cc:330] stopping tablet replica
I20260811 17:22:10.025017 32413 raft_consensus.cc:2229] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:22:10.025309 32413 raft_consensus.cc:2258] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:22:10.026546 32413 ts_tablet_manager.cc:1905] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:22:10.030521 32413 ts_tablet_manager.cc:1918] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:22:10.030658 32413 log.cc:1196] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/wal/wals/8b9c47e6f3994a1bb30641b3ffe1a02b
I20260811 17:22:10.030938 32413 ts_tablet_manager.cc:1939] T 8b9c47e6f3994a1bb30641b3ffe1a02b P 6308b38e7b554a12861b5e97ab5c0aa3: Deleting consensus metadata
I20260811 17:22:10.032927 32093 catalog_manager.cc:4841] TS 6308b38e7b554a12861b5e97ab5c0aa3 (127.9.157.129:32955): tablet 8b9c47e6f3994a1bb30641b3ffe1a02b (table test-table [id=d7a1027a35014c45b6733ce52cb649a5]) successfully deleted
W20260811 17:22:10.115094 32405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:22:10.115414 32405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:22:10.115463 32405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:22:10.121547 32405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:22:10.121640 32405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:22:10.121706 32405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:22:10.128760 32405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:39529
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=45189
--tserver_master_addrs=127.9.157.190:44901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:22:10.130034 32405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:22:10.131150 32405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:22:10.142179 32424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:10.142300 32419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:10.142302 32421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:10.142570 32405 server_base.cc:1034] running on GCE node
I20260811 17:22:10.143127 32405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:22:10.143968 32405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:22:10.145164 32405 hybrid_clock.cc:648] HybridClock initialized: now 1786468930145145 us; error 32 us; skew 500 ppm
Aug 11 17:22:10 dist-test-slave-mjjr krb5kdc[31253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468930, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:22:10.148439 32405 init.cc:377] Logged in from keytab as kudu/127.9.157.130@KRBTEST.COM (short username kudu)
I20260811 17:22:10.149439 32405 webserver.cc:460] Webserver started at http://127.9.157.130:45189/ using document root <none> and password file <none>
I20260811 17:22:10.150034 32405 fs_manager.cc:359] Metadata directory not provided
I20260811 17:22:10.150111 32405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:22:10.154711 32405 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:22:10.157160 32434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:10.158710 32405 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:22:10.158840 32405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "08aa841a1f38462ea7ab7c8fe54d9de0"
format_stamp: "Formatted at 2026-08-11 17:22:08 on dist-test-slave-mjjr"
server_key: "751ba72ecdfdef3dc7e873d35f9a9221"
server_key_iv: "f00062308485ddfbca68230f5b94525f"
server_key_version: "encryptionkey@0"
I20260811 17:22:10.159425 32405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:22:10.174526 32405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:22:10.178779 32405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:22:10.179021 32405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:22:10.179776 32405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:22:10.180742 32405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:22:10.180799 32405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:10.180845 32405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:22:10.180859 32405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:10.192360 32405 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:39529
I20260811 17:22:10.192378 32615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:39529 every 8 connection(s)
I20260811 17:22:10.193563 32405 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:22:10.199462 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 32405
I20260811 17:22:10.200021 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:40565
--local_ip_for_outbound_sockets=127.9.157.131
--tserver_master_addrs=127.9.157.190:44901
--webserver_port=35451
--webserver_interface=127.9.157.131
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 17:22:10 dist-test-slave-mjjr krb5kdc[31253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468930, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:22:10.212586 32404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:10.195582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:47249 (local address 127.9.157.190:44901)
0811 17:22:10.195832 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:10.195837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:10.196894 (+ 1057us) server_negotiation.cc:407] Connection header received
0811 17:22:10.198361 (+ 1467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:10.198365 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:10.198436 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:10.198547 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:10.200382 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.201248 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.203039 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.203291 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.203402 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:10.207116 (+ 3714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:10.207120 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:10.207122 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:10.207152 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:10.209102 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:10.210008 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:10.210011 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:10.210013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:10.210106 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:10.210772 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:10.210778 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:10.210780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:10.211143 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:10.211278 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:10.212259 (+ 981us) server_negotiation.cc:299] Negotiation successful
0811 17:22:10.212434 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260811 17:22:10.214020 32619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:10.196129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.130:47249)
0811 17:22:10.196682 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:10.196731 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:10.198104 (+ 1373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:10.198757 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:10.198765 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:10.199332 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:22:10.200160 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:10.200191 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.201428 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.201432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:10.202862 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:10.202880 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.203531 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.203536 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:10.203817 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:10.204539 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:22:10.204567 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:22:10.206848 (+ 2281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:22:10.209276 (+ 2428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:10.209287 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:10.209315 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:10.209830 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:10.210223 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:10.210229 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:10.210233 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:10.210488 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:10.211317 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:22:10.211326 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:22:10.211839 (+ 513us) client_negotiation.cc:769] Sending connection context
0811 17:22:10.212134 (+ 295us) client_negotiation.cc:240] Negotiation successful
0811 17:22:10.212617 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":146,"threads_started":1}
I20260811 17:22:10.215718 32616 heartbeater.cc:347] Connected to a master server at 127.9.157.190:44901
I20260811 17:22:10.216084 32616 heartbeater.cc:464] Registering TS with master...
I20260811 17:22:10.216814 32616 heartbeater.cc:511] Master 127.9.157.190:44901 requested a full tablet report, sending...
I20260811 17:22:10.218640 32111 ts_manager.cc:194] Registered new tserver with Master: 08aa841a1f38462ea7ab7c8fe54d9de0 (127.9.157.130:39529)
I20260811 17:22:10.219323 32111 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.130@KRBTEST.COM'} at 127.9.157.130:47249
W20260811 17:22:10.338115 32620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:22:10.338479 32620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:22:10.338568 32620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:22:10.343178 32620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:22:10.343333 32620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:22:10.343442 32620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:22:10.349717 32620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:40565
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=35451
--tserver_master_addrs=127.9.157.190:44901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:22:10.351076 32620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:22:10.352495 32620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:22:10.364253 32631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:10.364257 32628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:10.364257 32627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:10.364604 32620 server_base.cc:1034] running on GCE node
I20260811 17:22:10.365111 32620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:22:10.365916 32620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:22:10.367154 32620 hybrid_clock.cc:648] HybridClock initialized: now 1786468930367129 us; error 39 us; skew 500 ppm
Aug 11 17:22:10 dist-test-slave-mjjr krb5kdc[31253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468930, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:22:10.371052 32620 init.cc:377] Logged in from keytab as kudu/127.9.157.131@KRBTEST.COM (short username kudu)
I20260811 17:22:10.372336 32620 webserver.cc:460] Webserver started at http://127.9.157.131:35451/ using document root <none> and password file <none>
I20260811 17:22:10.373044 32620 fs_manager.cc:359] Metadata directory not provided
I20260811 17:22:10.373180 32620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:22:10.378140 32620 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 17:22:10.380787 32646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:10.382654 32620 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 17:22:10.382915 32620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "ceb96fcae8fc40b7a0d2f73592090af9"
format_stamp: "Formatted at 2026-08-11 17:22:09 on dist-test-slave-mjjr"
server_key: "7a0bb02280328a7f2dda16ac32d699f7"
server_key_iv: "e7b4e0cb44cb1647d22c73961f1fc1e1"
server_key_version: "encryptionkey@0"
I20260811 17:22:10.383582 32620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:22:10.399415 32620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:22:10.403559 32620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:22:10.403860 32620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:22:10.404924 32620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:22:10.405953 32620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:22:10.406032 32620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:10.406111 32620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:22:10.406155 32620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:10.417088 32620 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:40565
I20260811 17:22:10.417110 363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:40565 every 8 connection(s)
I20260811 17:22:10.418313 32620 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:22:10.420694 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 32620
Aug 11 17:22:10 dist-test-slave-mjjr krb5kdc[31253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468930, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:22:10.430869 369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:10.421459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42108 (local address 127.9.157.190:44901)
0811 17:22:10.421744 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:10.421750 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:10.421846 (+ 96us) server_negotiation.cc:407] Connection header received
0811 17:22:10.422054 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:10.422057 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:10.422119 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:10.422219 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:10.423054 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.423761 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.424980 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.425220 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.425317 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:10.427284 (+ 1967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:10.427287 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:10.427289 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:10.427315 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:10.429037 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:10.429508 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:10.429515 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:10.429518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:10.429611 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:10.429890 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:10.429894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:10.429895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:10.430169 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:10.430334 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:10.430575 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 17:22:10.430688 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":114,"threads_started":1}
I20260811 17:22:10.436079 32404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:10.420673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:46725 (local address 127.9.157.190:44901)
0811 17:22:10.420846 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:10.420851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:10.422054 (+ 1203us) server_negotiation.cc:407] Connection header received
0811 17:22:10.423498 (+ 1444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:10.423501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:10.423552 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:10.423682 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:10.425409 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.426160 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.428002 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.428372 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.428503 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:10.431795 (+ 3292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:10.431801 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:10.431805 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:10.431834 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:10.433956 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:10.434542 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:10.434545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:10.434546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:10.434623 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:10.434953 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:10.434955 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:10.434956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:10.435143 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:10.435219 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:10.435823 (+ 604us) server_negotiation.cc:299] Negotiation successful
0811 17:22:10.435941 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":64}
I20260811 17:22:10.437093 368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:10.421210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.131:46725)
0811 17:22:10.421910 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:10.421965 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:10.423279 (+ 1314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:10.423988 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:10.423999 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:10.424552 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:22:10.425221 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:10.425238 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.426325 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.426330 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:10.427864 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:10.427878 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.428574 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.428577 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:10.428741 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:10.429306 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:22:10.429325 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:22:10.431541 (+ 2216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:22:10.434083 (+ 2542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:10.434089 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:10.434106 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:10.434434 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:10.434705 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:10.434707 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:10.434708 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:10.434859 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:10.435234 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:22:10.435240 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:22:10.435520 (+ 280us) client_negotiation.cc:769] Sending connection context
0811 17:22:10.435766 (+ 246us) client_negotiation.cc:240] Negotiation successful
0811 17:22:10.436098 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":185,"threads_started":1}
I20260811 17:22:10.438530 365 heartbeater.cc:347] Connected to a master server at 127.9.157.190:44901
I20260811 17:22:10.438876 365 heartbeater.cc:464] Registering TS with master...
I20260811 17:22:10.439734 365 heartbeater.cc:511] Master 127.9.157.190:44901 requested a full tablet report, sending...
I20260811 17:22:10.441080 32111 ts_manager.cc:194] Registered new tserver with Master: ceb96fcae8fc40b7a0d2f73592090af9 (127.9.157.131:40565)
I20260811 17:22:10.441716 32111 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.131@KRBTEST.COM'} at 127.9.157.131:46725
I20260811 17:22:10.445173 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:22:10.449663 32404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:10.445864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42122 (local address 127.9.157.190:44901)
0811 17:22:10.446002 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:10.446006 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:10.446210 (+ 204us) server_negotiation.cc:407] Connection header received
0811 17:22:10.446360 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:10.446363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:10.446490 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:10.446585 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:22:10.446805 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.447318 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.448252 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.448477 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.448554 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:10.448845 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:22:10.449327 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:22:10.449436 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:10.449488 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 17:22:10.449542 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 17:22:10.450430 32111 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:42122:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:22:10.454203 32111 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:22:10.463148 32289 tablet_service.cc:1467] Processing CreateTablet for tablet 2e99668915324cd9962e40a7e97ff179 (DEFAULT_TABLE table=test-table [id=c32620f7924f40499f2d7a0cc62d9449]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:22:10.463974 32289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e99668915324cd9962e40a7e97ff179. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:22:10.467186 32225 tablet_bootstrap.cc:492] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3: Bootstrap starting.
I20260811 17:22:10.467579 32411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:10.461916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:39529 (local address 127.0.0.1:44652)
0811 17:22:10.462084 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:10.462100 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:10.462194 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:10.463283 (+ 1089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:10.463286 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:10.463308 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:22:10.463446 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:10.463456 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.465023 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.465026 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:10.466153 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:10.466163 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.467115 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.467119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:10.467228 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:10.467283 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 17:22:10.467340 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 17:22:10.467430 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":5}
I20260811 17:22:10.468235 32225 tablet_bootstrap.cc:654] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3: Neither blocks nor log segments found. Creating new log.
I20260811 17:22:10.468638 376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:10.462141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44652 (local address 127.9.157.130:39529)
0811 17:22:10.462603 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:10.462618 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:10.462647 (+ 29us) server_negotiation.cc:407] Connection header received
0811 17:22:10.462738 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:10.462744 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:10.463056 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:10.463287 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:22:10.463600 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.464843 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.466293 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.466916 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.468082 (+ 1166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:10.468263 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:10.468356 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 17:22:10.468491 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":175,"threads_started":1}
I20260811 17:22:10.468662 379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:10.462853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53386 (local address 127.9.157.131:40565)
0811 17:22:10.463428 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:10.463441 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:10.463466 (+ 25us) server_negotiation.cc:407] Connection header received
0811 17:22:10.463548 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:10.463555 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:10.463784 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:10.464161 (+ 377us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:22:10.464377 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.465531 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.467032 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.468043 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.468145 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:10.468263 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:10.468449 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 17:22:10.468531 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":157,"threads_started":1}
I20260811 17:22:10.468662 377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:10.462460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:40565 (local address 127.0.0.1:53386)
0811 17:22:10.462930 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:10.462945 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:10.463059 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:10.463942 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:10.463945 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:10.463965 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:22:10.464092 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:10.464100 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.465697 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.465704 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:10.466871 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:10.466885 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.468156 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.468162 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:10.468278 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:10.468308 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 17:22:10.468398 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 17:22:10.468485 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":109,"threads_started":1}
I20260811 17:22:10.471774 32719 tablet_service.cc:1467] Processing CreateTablet for tablet 2e99668915324cd9962e40a7e97ff179 (DEFAULT_TABLE table=test-table [id=c32620f7924f40499f2d7a0cc62d9449]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:22:10.471771 32506 tablet_service.cc:1467] Processing CreateTablet for tablet 2e99668915324cd9962e40a7e97ff179 (DEFAULT_TABLE table=test-table [id=c32620f7924f40499f2d7a0cc62d9449]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:22:10.472992 32506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e99668915324cd9962e40a7e97ff179. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:22:10.473044 32719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e99668915324cd9962e40a7e97ff179. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:22:10.473635 32225 tablet_bootstrap.cc:492] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3: No bootstrap required, opened a new log
I20260811 17:22:10.473758 32225 ts_tablet_manager.cc:1397] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260811 17:22:10.474313 32225 raft_consensus.cc:348] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } }
I20260811 17:22:10.474475 32225 raft_consensus.cc:374] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:22:10.474540 32225 raft_consensus.cc:729] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6308b38e7b554a12861b5e97ab5c0aa3, State: Initialized, Role: FOLLOWER
I20260811 17:22:10.474685 32225 consensus_queue.cc:260] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } }
I20260811 17:22:10.474985 32225 ts_tablet_manager.cc:1428] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
W20260811 17:22:10.476761 32402 tablet.cc:2369] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:22:10.481797 385 tablet_bootstrap.cc:492] T 2e99668915324cd9962e40a7e97ff179 P ceb96fcae8fc40b7a0d2f73592090af9: Bootstrap starting.
I20260811 17:22:10.482183 384 tablet_bootstrap.cc:492] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0: Bootstrap starting.
I20260811 17:22:10.484625 384 tablet_bootstrap.cc:654] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0: Neither blocks nor log segments found. Creating new log.
I20260811 17:22:10.484671 385 tablet_bootstrap.cc:654] T 2e99668915324cd9962e40a7e97ff179 P ceb96fcae8fc40b7a0d2f73592090af9: Neither blocks nor log segments found. Creating new log.
I20260811 17:22:10.485499 385 log.cc:824] T 2e99668915324cd9962e40a7e97ff179 P ceb96fcae8fc40b7a0d2f73592090af9: Log is configured to *not* fsync() on all Append() calls
I20260811 17:22:10.485499 384 log.cc:824] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0: Log is configured to *not* fsync() on all Append() calls
I20260811 17:22:10.487973 384 tablet_bootstrap.cc:492] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0: No bootstrap required, opened a new log
I20260811 17:22:10.487986 385 tablet_bootstrap.cc:492] T 2e99668915324cd9962e40a7e97ff179 P ceb96fcae8fc40b7a0d2f73592090af9: No bootstrap required, opened a new log
I20260811 17:22:10.488214 385 ts_tablet_manager.cc:1397] T 2e99668915324cd9962e40a7e97ff179 P ceb96fcae8fc40b7a0d2f73592090af9: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 17:22:10.488222 384 ts_tablet_manager.cc:1397] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 17:22:10.491375 384 raft_consensus.cc:348] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } }
I20260811 17:22:10.491375 385 raft_consensus.cc:348] T 2e99668915324cd9962e40a7e97ff179 P ceb96fcae8fc40b7a0d2f73592090af9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } }
I20260811 17:22:10.491741 384 raft_consensus.cc:374] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:22:10.491768 385 raft_consensus.cc:374] T 2e99668915324cd9962e40a7e97ff179 P ceb96fcae8fc40b7a0d2f73592090af9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:22:10.491832 385 raft_consensus.cc:729] T 2e99668915324cd9962e40a7e97ff179 P ceb96fcae8fc40b7a0d2f73592090af9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb96fcae8fc40b7a0d2f73592090af9, State: Initialized, Role: FOLLOWER
I20260811 17:22:10.491844 384 raft_consensus.cc:729] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08aa841a1f38462ea7ab7c8fe54d9de0, State: Initialized, Role: FOLLOWER
I20260811 17:22:10.492393 385 consensus_queue.cc:260] T 2e99668915324cd9962e40a7e97ff179 P ceb96fcae8fc40b7a0d2f73592090af9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } }
I20260811 17:22:10.492436 384 consensus_queue.cc:260] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } }
I20260811 17:22:10.493449 32616 heartbeater.cc:503] Master 127.9.157.190:44901 was elected leader, sending a full tablet report...
I20260811 17:22:10.493716 384 ts_tablet_manager.cc:1428] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0: Time spent starting tablet: real 0.005s user 0.002s sys 0.004s
I20260811 17:22:10.493765 385 ts_tablet_manager.cc:1428] T 2e99668915324cd9962e40a7e97ff179 P ceb96fcae8fc40b7a0d2f73592090af9: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 17:22:10.494189 365 heartbeater.cc:503] Master 127.9.157.190:44901 was elected leader, sending a full tablet report...
I20260811 17:22:10.667428 390 raft_consensus.cc:482] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:22:10.667734 390 raft_consensus.cc:504] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } }
I20260811 17:22:10.669015 390 leader_election.cc:288] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ceb96fcae8fc40b7a0d2f73592090af9 (127.9.157.131:40565), 6308b38e7b554a12861b5e97ab5c0aa3 (127.9.157.129:32955)
W20260811 17:22:10.669814 366 tablet.cc:2369] T 2e99668915324cd9962e40a7e97ff179 P ceb96fcae8fc40b7a0d2f73592090af9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:22:10.673380 32619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:10.669249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:40565 (local address 127.9.157.130:56723)
0811 17:22:10.669383 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:10.669402 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:10.669505 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:10.669923 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:10.669927 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:10.670007 (+ 80us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:22:10.670136 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:10.670144 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.670979 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.670982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:10.672026 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:10.672036 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.672891 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.672895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:10.673008 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:10.673036 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 17:22:10.673121 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 17:22:10.673202 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"spinlock_wait_cycles":115072}
I20260811 17:22:10.673372 379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:10.669479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:56723 (local address 127.9.157.131:40565)
0811 17:22:10.669623 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:10.669629 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:10.669643 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:22:10.669702 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:10.669705 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:10.669754 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:10.669861 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:22:10.670272 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.670854 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.672160 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.672771 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.672873 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:10.672919 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:10.673172 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 17:22:10.673251 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 17:22:10.674041 391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:10.669351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:32955 (local address 127.9.157.130:42441)
0811 17:22:10.669771 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:10.669787 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:10.670001 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:10.670406 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:10.670409 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:10.670425 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:22:10.670552 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:10.670558 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.671495 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.671498 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:10.672536 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:10.672547 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.673515 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.673519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:10.673641 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:10.673679 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 17:22:10.673805 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 17:22:10.673890 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":118,"threads_started":1}
I20260811 17:22:10.674077 392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:10.669699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:42441 (local address 127.9.157.129:32955)
0811 17:22:10.670100 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:10.670106 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:10.670119 (+ 13us) server_negotiation.cc:407] Connection header received
0811 17:22:10.670175 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:10.670181 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:10.670260 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:10.670392 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:22:10.670669 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.671368 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.672669 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.673350 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.673486 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:10.673666 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:10.673853 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 17:22:10.673943 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":97,"threads_started":1}
I20260811 17:22:10.674356 32759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e99668915324cd9962e40a7e97ff179" candidate_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" is_pre_election: true
I20260811 17:22:10.674741 32759 raft_consensus.cc:2454] T 2e99668915324cd9962e40a7e97ff179 P ceb96fcae8fc40b7a0d2f73592090af9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08aa841a1f38462ea7ab7c8fe54d9de0 in term 0.
I20260811 17:22:10.674760 32319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e99668915324cd9962e40a7e97ff179" candidate_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" is_pre_election: true
I20260811 17:22:10.675096 32319 raft_consensus.cc:2454] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08aa841a1f38462ea7ab7c8fe54d9de0 in term 0.
I20260811 17:22:10.675500 32439 leader_election.cc:302] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08aa841a1f38462ea7ab7c8fe54d9de0, ceb96fcae8fc40b7a0d2f73592090af9; no voters:
I20260811 17:22:10.675926 390 raft_consensus.cc:2781] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:22:10.676043 390 raft_consensus.cc:482] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:22:10.676110 390 raft_consensus.cc:3037] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:22:10.677376 390 raft_consensus.cc:504] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } }
I20260811 17:22:10.677801 390 leader_election.cc:288] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0 [CANDIDATE]: Term 1 election: Requested vote from peers ceb96fcae8fc40b7a0d2f73592090af9 (127.9.157.131:40565), 6308b38e7b554a12861b5e97ab5c0aa3 (127.9.157.129:32955)
I20260811 17:22:10.678305 32759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e99668915324cd9962e40a7e97ff179" candidate_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceb96fcae8fc40b7a0d2f73592090af9"
I20260811 17:22:10.678481 32759 raft_consensus.cc:3037] T 2e99668915324cd9962e40a7e97ff179 P ceb96fcae8fc40b7a0d2f73592090af9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:22:10.678572 32319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e99668915324cd9962e40a7e97ff179" candidate_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6308b38e7b554a12861b5e97ab5c0aa3"
I20260811 17:22:10.678771 32319 raft_consensus.cc:3037] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:22:10.679842 32759 raft_consensus.cc:2454] T 2e99668915324cd9962e40a7e97ff179 P ceb96fcae8fc40b7a0d2f73592090af9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08aa841a1f38462ea7ab7c8fe54d9de0 in term 1.
I20260811 17:22:10.680171 32319 raft_consensus.cc:2454] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08aa841a1f38462ea7ab7c8fe54d9de0 in term 1.
I20260811 17:22:10.680297 32439 leader_election.cc:302] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08aa841a1f38462ea7ab7c8fe54d9de0, ceb96fcae8fc40b7a0d2f73592090af9; no voters:
I20260811 17:22:10.680569 390 raft_consensus.cc:2781] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:22:10.680860 390 raft_consensus.cc:686] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0 [term 1 LEADER]: Becoming Leader. State: Replica: 08aa841a1f38462ea7ab7c8fe54d9de0, State: Running, Role: LEADER
I20260811 17:22:10.681308 390 consensus_queue.cc:237] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } }
I20260811 17:22:10.685029 32110 catalog_manager.cc:5495] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 08aa841a1f38462ea7ab7c8fe54d9de0 (127.9.157.130). New cstate: current_term: 1 leader_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } health_report { overall_health: UNKNOWN } } }
W20260811 17:22:10.694897 32618 tablet.cc:2369] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:22:10.719054 376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:10.714971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44656 (local address 127.9.157.130:39529)
0811 17:22:10.715128 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:10.715133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:10.715149 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:22:10.715276 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:10.715278 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:10.715341 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:10.715474 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:22:10.715830 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.716417 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.717343 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:10.717588 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:10.717680 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:10.718067 (+ 387us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:22:10.718564 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:22:10.718673 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:10.718788 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 17:22:10.718869 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 17:22:10.726125 32759 raft_consensus.cc:1261] T 2e99668915324cd9962e40a7e97ff179 P ceb96fcae8fc40b7a0d2f73592090af9 [term 1 FOLLOWER]: Refusing update from remote peer 08aa841a1f38462ea7ab7c8fe54d9de0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:22:10.726087 32319 raft_consensus.cc:1261] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [term 1 FOLLOWER]: Refusing update from remote peer 08aa841a1f38462ea7ab7c8fe54d9de0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:22:10.726956 393 consensus_queue.cc:1035] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:22:10.727185 390 consensus_queue.cc:1035] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:22:10.737252 408 mvcc.cc:205] Tried to move back new op lower bound from 7317376740245217280 to 7317376740073975808. Current Snapshot: MvccSnapshot[applied={T|T < 7317376740245217280}]
I20260811 17:22:10.739897 403 mvcc.cc:205] Tried to move back new op lower bound from 7317376740245217280 to 7317376740073975808. Current Snapshot: MvccSnapshot[applied={T|T < 7317376740245217280 or (T in {7317376740245217280})}]
I20260811 17:22:10.750020 32111 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:42122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:22:10.750233 32111 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:42122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:22:10.753151 32111 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0: Sending DeleteTablet for 3 replicas of tablet 2e99668915324cd9962e40a7e97ff179
I20260811 17:22:10.754112 32289 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e99668915324cd9962e40a7e97ff179 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:22:10 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:42918
I20260811 17:22:10.754176 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 32165
I20260811 17:22:10.754146 32719 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e99668915324cd9962e40a7e97ff179 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:22:10 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:53386
I20260811 17:22:10.757138 32506 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e99668915324cd9962e40a7e97ff179 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:22:10 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:44652
I20260811 17:22:10.759591 421 tablet_replica.cc:330] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0: stopping tablet replica
I20260811 17:22:10.760071 421 raft_consensus.cc:2229] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0 [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:22:10.760604 421 raft_consensus.cc:2258] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:22:10.761087 419 tablet_replica.cc:330] T 2e99668915324cd9962e40a7e97ff179 P ceb96fcae8fc40b7a0d2f73592090af9: stopping tablet replica
I20260811 17:22:10.761420 419 raft_consensus.cc:2229] T 2e99668915324cd9962e40a7e97ff179 P ceb96fcae8fc40b7a0d2f73592090af9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:22:10.761760 419 raft_consensus.cc:2258] T 2e99668915324cd9962e40a7e97ff179 P ceb96fcae8fc40b7a0d2f73592090af9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:22:10.766960 419 ts_tablet_manager.cc:1905] T 2e99668915324cd9962e40a7e97ff179 P ceb96fcae8fc40b7a0d2f73592090af9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:22:10.770282 419 ts_tablet_manager.cc:1918] T 2e99668915324cd9962e40a7e97ff179 P ceb96fcae8fc40b7a0d2f73592090af9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:22:10.770442 419 log.cc:1196] T 2e99668915324cd9962e40a7e97ff179 P ceb96fcae8fc40b7a0d2f73592090af9: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/wal/wals/2e99668915324cd9962e40a7e97ff179
I20260811 17:22:10.770860 419 ts_tablet_manager.cc:1939] T 2e99668915324cd9962e40a7e97ff179 P ceb96fcae8fc40b7a0d2f73592090af9: Deleting consensus metadata
I20260811 17:22:10.771679 421 ts_tablet_manager.cc:1905] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:22:10.772432 32092 catalog_manager.cc:4841] TS ceb96fcae8fc40b7a0d2f73592090af9 (127.9.157.131:40565): tablet 2e99668915324cd9962e40a7e97ff179 (table test-table [id=c32620f7924f40499f2d7a0cc62d9449]) successfully deleted
I20260811 17:22:10.775101 421 ts_tablet_manager.cc:1918] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:22:10.775251 421 log.cc:1196] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/wal/wals/2e99668915324cd9962e40a7e97ff179
I20260811 17:22:10.775717 421 ts_tablet_manager.cc:1939] T 2e99668915324cd9962e40a7e97ff179 P 08aa841a1f38462ea7ab7c8fe54d9de0: Deleting consensus metadata
I20260811 17:22:10.777153 32092 catalog_manager.cc:4841] TS 08aa841a1f38462ea7ab7c8fe54d9de0 (127.9.157.130:39529): tablet 2e99668915324cd9962e40a7e97ff179 (table test-table [id=c32620f7924f40499f2d7a0cc62d9449]) successfully deleted
W20260811 17:22:10.782714 32093 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.157.129:32955 (error 108)
I20260811 17:22:10.782920 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 32405
I20260811 17:22:10.792346 377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:10.791690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:32955 (local address 127.0.0.1:42928)
0811 17:22:10.792092 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:10.792204 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.129:32955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277}
W20260811 17:22:10.792655 32093 catalog_manager.cc:4584] TS 6308b38e7b554a12861b5e97ab5c0aa3 (127.9.157.129:32955): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2e99668915324cd9962e40a7e97ff179: Network error: Client connection negotiation failed: client connection to 127.9.157.129:32955: connect: Connection refused (error 111)
I20260811 17:22:10.805034 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 32620
I20260811 17:22:10.823923 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 32063
I20260811 17:22:10.836179 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:44901
--webserver_interface=127.9.157.190
--webserver_port=37181
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:44901
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:22:10.966363 423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:22:10.966722 423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:22:10.966768 423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:22:10.971114 423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:22:10.971213 423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:22:10.971235 423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:22:10.971251 423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:22:10.971267 423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:22:10.977280 423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:44901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:44901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=37181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:22:10.978605 423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:22:10.979875 423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:22:10.989333 428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:10.989506 429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:10.989333 433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:10.989666 423 server_base.cc:1034] running on GCE node
I20260811 17:22:10.990424 423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:22:10.991688 423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:22:10.992923 423 hybrid_clock.cc:648] HybridClock initialized: now 1786468930992888 us; error 55 us; skew 500 ppm
Aug 11 17:22:10 dist-test-slave-mjjr krb5kdc[31253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468930, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:22:10.996258 423 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:22:10.997476 423 webserver.cc:460] Webserver started at http://127.9.157.190:37181/ using document root <none> and password file <none>
I20260811 17:22:10.998183 423 fs_manager.cc:359] Metadata directory not provided
I20260811 17:22:10.998311 423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:22:11.003088 423 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:22:11.005824 446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:11.007414 423 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:22:11.007618 423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0"
format_stamp: "Formatted at 2026-08-11 17:22:08 on dist-test-slave-mjjr"
server_key: "b1e218ce3e3d12af42b137bf90cf6fc5"
server_key_iv: "89e7f2fe7ff6fcfcdd4e86584a1b38a6"
server_key_version: "encryptionkey@0"
I20260811 17:22:11.008327 423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:22:11.037922 423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:22:11.041690 423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:22:11.041971 423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:22:11.050477 423 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:44901
I20260811 17:22:11.050798 525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:44901 every 8 connection(s)
I20260811 17:22:11.051836 423 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:22:11.055274 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 423
I20260811 17:22:11.055826 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:32955
--local_ip_for_outbound_sockets=127.9.157.129
--tserver_master_addrs=127.9.157.190:44901
--webserver_port=33223
--webserver_interface=127.9.157.129
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:22:11.057612 526 sys_catalog.cc:263] Verifying existing consensus state
I20260811 17:22:11.059773 526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0: Bootstrap starting.
I20260811 17:22:11.080843 526 log.cc:824] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0: Log is configured to *not* fsync() on all Append() calls
I20260811 17:22:11.094473 526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 17:22:11.094998 526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0: Bootstrap complete.
I20260811 17:22:11.099627 526 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44901 } }
I20260811 17:22:11.100467 526 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab1d7cfe1fe9459b9e20d6139e27a4b0, State: Initialized, Role: FOLLOWER
I20260811 17:22:11.101154 526 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44901 } }
I20260811 17:22:11.101296 526 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:22:11.101389 526 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:22:11.101505 526 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [term 2 FOLLOWER]: Advancing to term 3
I20260811 17:22:11.103964 526 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44901 } }
I20260811 17:22:11.104363 526 leader_election.cc:302] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab1d7cfe1fe9459b9e20d6139e27a4b0; no voters:
I20260811 17:22:11.104765 526 leader_election.cc:288] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 17:22:11.104864 531 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 17:22:11.105093 531 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [term 3 LEADER]: Becoming Leader. State: Replica: ab1d7cfe1fe9459b9e20d6139e27a4b0, State: Running, Role: LEADER
I20260811 17:22:11.105433 531 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44901 } }
I20260811 17:22:11.106230 526 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:22:11.107067 532 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44901 } } }
I20260811 17:22:11.107265 532 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [sys.catalog]: This master's current role is: LEADER
I20260811 17:22:11.107967 533 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab1d7cfe1fe9459b9e20d6139e27a4b0. Latest consensus state: current_term: 3 leader_uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab1d7cfe1fe9459b9e20d6139e27a4b0" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44901 } } }
I20260811 17:22:11.108147 533 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab1d7cfe1fe9459b9e20d6139e27a4b0 [sys.catalog]: This master's current role is: LEADER
I20260811 17:22:11.108335 542 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:22:11.113687 542 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:22:11.114400 542 catalog_manager.cc:1259] Loaded cluster ID: 74fa366ef4d14041a98b9ee9435ddc01
I20260811 17:22:11.114494 542 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 17:22:11.115559 542 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fd5a03639ad google::LogMessage::Fail() at ??:0
@ 0x7fd5a036688a google::LogMessage::SendToLog() at ??:0
@ 0x7fd5a03634de google::LogMessage::Flush() at ??:0
@ 0x7fd5a0364a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fd5aaca2fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fd5aaca3d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fd5aaca0579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fd5aace4640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fd5a25b53d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fd5a25b5c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fd5a25b73f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fd5a25a6512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fd5a3bea6db start_thread at ??:0
@ 0x7fd59ee6161f clone at ??:0
W20260811 17:22:11.225095 528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:22:11.225582 528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:22:11.225651 528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:22:11.232755 528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:22:11.232872 528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:22:11.232975 528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:22:11.242792 528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:32955
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=33223
--tserver_master_addrs=127.9.157.190:44901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:22:11.244518 528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:22:11.246186 528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:22:11.259603 585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:11.259609 589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:11.259603 586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:11.260124 528 server_base.cc:1034] running on GCE node
I20260811 17:22:11.260613 528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:22:11.261426 528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:22:11.262650 528 hybrid_clock.cc:648] HybridClock initialized: now 1786468931262620 us; error 47 us; skew 500 ppm
Aug 11 17:22:11 dist-test-slave-mjjr krb5kdc[31253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468931, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:22:11.265967 528 init.cc:377] Logged in from keytab as kudu/127.9.157.129@KRBTEST.COM (short username kudu)
I20260811 17:22:11.267048 528 webserver.cc:460] Webserver started at http://127.9.157.129:33223/ using document root <none> and password file <none>
I20260811 17:22:11.267776 528 fs_manager.cc:359] Metadata directory not provided
I20260811 17:22:11.267854 528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:22:11.272281 528 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:22:11.274982 599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:11.276598 528 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:22:11.276746 528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "6308b38e7b554a12861b5e97ab5c0aa3"
format_stamp: "Formatted at 2026-08-11 17:22:08 on dist-test-slave-mjjr"
server_key: "714c6f8b86c4ddec405bdc59cbdb79aa"
server_key_iv: "d9bfd631d23c0920869b56b086a46857"
server_key_version: "encryptionkey@0"
I20260811 17:22:11.277350 528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:22:11.298213 528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:22:11.302666 528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:22:11.303184 528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:22:11.304059 528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:22:11.305476 611 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 17:22:11.312597 528 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 17:22:11.312740 528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.008s user 0.001s sys 0.000s
I20260811 17:22:11.312853 528 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 17:22:11.314934 528 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 17:22:11.315033 528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 17:22:11.315394 611 tablet_bootstrap.cc:492] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3: Bootstrap starting.
I20260811 17:22:11.326803 528 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:32955
I20260811 17:22:11.328092 528 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:22:11.331915 773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:32955 every 8 connection(s)
I20260811 17:22:11.336495 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 528
I20260811 17:22:11.336962 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:39529
--local_ip_for_outbound_sockets=127.9.157.130
--tserver_master_addrs=127.9.157.190:44901
--webserver_port=45189
--webserver_interface=127.9.157.130
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:22:11.340616 611 log.cc:824] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3: Log is configured to *not* fsync() on all Append() calls
I20260811 17:22:11.352047 611 tablet_bootstrap.cc:492] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 17:22:11.352711 611 tablet_bootstrap.cc:492] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3: Bootstrap complete.
I20260811 17:22:11.354530 611 ts_tablet_manager.cc:1397] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3: Time spent bootstrapping tablet: real 0.039s user 0.029s sys 0.007s
I20260811 17:22:11.358314 611 raft_consensus.cc:348] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } }
I20260811 17:22:11.359040 611 raft_consensus.cc:729] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6308b38e7b554a12861b5e97ab5c0aa3, State: Initialized, Role: FOLLOWER
I20260811 17:22:11.359766 611 consensus_queue.cc:260] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } }
I20260811 17:22:11.361097 611 ts_tablet_manager.cc:1428] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
W20260811 17:22:11.435139 778 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:22:11.334503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.129:32965)
0811 17:22:11.335074 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:11.335132 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:11.336067 (+ 935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:11.434039 (+ 97972us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":187,"threads_started":1}
W20260811 17:22:11.435868 775 heartbeater.cc:644] Failed to heartbeat to 127.9.157.190:44901 (0 consecutive failures): Network error: Failed to ping master at 127.9.157.190:44901: Client connection negotiation failed: client connection to 127.9.157.190:44901: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 17:22:11.436842 778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:11.436473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.129:60765)
0811 17:22:11.436645 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:11.436749 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 17:22:11.437785 778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:11.437503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.129:56549)
0811 17:22:11.437647 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:11.437703 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 17:22:11.438041 775 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 17:22:11.469591 779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:22:11.469910 779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:22:11.469957 779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:22:11.474316 779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:22:11.474419 779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:22:11.474485 779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:22:11.480384 779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:39529
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=45189
--tserver_master_addrs=127.9.157.190:44901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:22:11.481766 779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:22:11.482998 779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:22:11.494472 779 server_base.cc:1034] running on GCE node
W20260811 17:22:11.494393 792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:11.494433 789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:11.494410 790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:11.495213 779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:22:11.496290 779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:22:11.497510 779 hybrid_clock.cc:648] HybridClock initialized: now 1786468931497472 us; error 62 us; skew 500 ppm
Aug 11 17:22:11 dist-test-slave-mjjr krb5kdc[31253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468931, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:22:11.505093 779 init.cc:377] Logged in from keytab as kudu/127.9.157.130@KRBTEST.COM (short username kudu)
I20260811 17:22:11.506253 779 webserver.cc:460] Webserver started at http://127.9.157.130:45189/ using document root <none> and password file <none>
I20260811 17:22:11.506981 779 fs_manager.cc:359] Metadata directory not provided
I20260811 17:22:11.507095 779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:22:11.512215 779 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:22:11.514828 803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:11.516295 779 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:22:11.516499 779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "08aa841a1f38462ea7ab7c8fe54d9de0"
format_stamp: "Formatted at 2026-08-11 17:22:08 on dist-test-slave-mjjr"
server_key: "751ba72ecdfdef3dc7e873d35f9a9221"
server_key_iv: "f00062308485ddfbca68230f5b94525f"
server_key_version: "encryptionkey@0"
I20260811 17:22:11.517177 779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:22:11.539598 779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:22:11.543176 779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:22:11.543447 779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:22:11.544185 779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:22:11.545140 779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:22:11.545229 779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:11.545301 779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:22:11.545367 779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:11.555771 779 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:39529
I20260811 17:22:11.555792 988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:39529 every 8 connection(s)
I20260811 17:22:11.557008 779 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:22:11.560050 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 779
I20260811 17:22:11.560616 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:40565
--local_ip_for_outbound_sockets=127.9.157.131
--tserver_master_addrs=127.9.157.190:44901
--webserver_port=35451
--webserver_interface=127.9.157.131
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:22:11.562183 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:11.559591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.130:50413)
0811 17:22:11.560521 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:11.560655 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":194,"threads_started":1}
W20260811 17:22:11.562767 989 heartbeater.cc:644] Failed to heartbeat to 127.9.157.190:44901 (0 consecutive failures): Network error: Failed to ping master at 127.9.157.190:44901: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
I20260811 17:22:11.563738 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:11.563402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.130:50489)
0811 17:22:11.563553 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:11.563619 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 17:22:11.564576 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:11.564325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.130:60355)
0811 17:22:11.564463 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:11.564516 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 17:22:11.564764 989 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 17:22:11.691035 992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:22:11.691380 992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:22:11.691465 992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:22:11.695794 992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:22:11.695914 992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:22:11.696024 992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:22:11.701761 992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:40565
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=35451
--tserver_master_addrs=127.9.157.190:44901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:22:11.703066 992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:22:11.704262 992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:22:11.715004 1005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:11.715112 1001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:11.715093 992 server_base.cc:1034] running on GCE node
W20260811 17:22:11.715013 1002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:11.715919 992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:22:11.716702 992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:22:11.717968 992 hybrid_clock.cc:648] HybridClock initialized: now 1786468931717928 us; error 55 us; skew 500 ppm
Aug 11 17:22:11 dist-test-slave-mjjr krb5kdc[31253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468931, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:22:11.721467 992 init.cc:377] Logged in from keytab as kudu/127.9.157.131@KRBTEST.COM (short username kudu)
I20260811 17:22:11.722734 992 webserver.cc:460] Webserver started at http://127.9.157.131:35451/ using document root <none> and password file <none>
I20260811 17:22:11.723452 992 fs_manager.cc:359] Metadata directory not provided
I20260811 17:22:11.723573 992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:22:11.728164 992 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 17:22:11.730809 1014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:11.732478 992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:22:11.732656 992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "ceb96fcae8fc40b7a0d2f73592090af9"
format_stamp: "Formatted at 2026-08-11 17:22:09 on dist-test-slave-mjjr"
server_key: "7a0bb02280328a7f2dda16ac32d699f7"
server_key_iv: "e7b4e0cb44cb1647d22c73961f1fc1e1"
server_key_version: "encryptionkey@0"
I20260811 17:22:11.733314 992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:22:11.762739 992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:22:11.766160 992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:22:11.766423 992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:22:11.767125 992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:22:11.768095 992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:22:11.768175 992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:11.768261 992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:22:11.768316 992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:11.778620 992 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:40565
I20260811 17:22:11.778642 1187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:40565 every 8 connection(s)
I20260811 17:22:11.779807 992 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:22:11.780977 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 992
I20260811 17:22:11.784958 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:11.782678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.131:40007)
0811 17:22:11.783429 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:11.783558 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":241,"threads_started":1}
W20260811 17:22:11.785596 1188 heartbeater.cc:644] Failed to heartbeat to 127.9.157.190:44901 (0 consecutive failures): Network error: Failed to ping master at 127.9.157.190:44901: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
I20260811 17:22:11.786365 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:11.786098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.131:47545)
0811 17:22:11.786243 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:11.786300 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 17:22:11.787034 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:11.786826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.131:53943)
0811 17:22:11.786938 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:11.786981 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 17:22:11.787245 1188 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 17:22:12.439332 1193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:12.438811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.129:36671)
0811 17:22:12.439133 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:12.439200 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":97,"threads_started":1}
I20260811 17:22:12.566154 1194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:12.565606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.130:39409)
0811 17:22:12.565935 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:12.566008 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":92,"threads_started":1}
I20260811 17:22:12.613662 1196 raft_consensus.cc:482] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:22:12.613921 1196 raft_consensus.cc:504] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } }
I20260811 17:22:12.615226 1196 leader_election.cc:288] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 08aa841a1f38462ea7ab7c8fe54d9de0 (127.9.157.130:39529), ceb96fcae8fc40b7a0d2f73592090af9 (127.9.157.131:40565)
Aug 11 17:22:12 dist-test-slave-mjjr krb5kdc[31253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468931, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for kudu/127.9.157.131@KRBTEST.COM
Aug 11 17:22:12 dist-test-slave-mjjr krb5kdc[31253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468931, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for kudu/127.9.157.130@KRBTEST.COM
I20260811 17:22:12.631839 1198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:12.615808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:40565 (local address 127.9.157.129:36517)
0811 17:22:12.616150 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:12.616186 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:12.616311 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:12.617813 (+ 1502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:12.617820 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:12.618443 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:22:12.619035 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:12.619054 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:12.620581 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:12.620584 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:12.621754 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:12.621767 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:12.622652 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:12.622656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:12.622975 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:12.623726 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:22:12.623762 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:22:12.626020 (+ 2258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:22:12.629301 (+ 3281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:12.629307 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:12.629310 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:12.629539 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:12.629937 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:12.629941 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:12.629943 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:12.630037 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:12.630711 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:22:12.630720 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:22:12.630984 (+ 264us) client_negotiation.cc:769] Sending connection context
0811 17:22:12.631333 (+ 349us) client_negotiation.cc:240] Negotiation successful
0811 17:22:12.631591 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":179,"spinlock_wait_cycles":1920,"thread_start_us":89,"threads_started":1}
I20260811 17:22:12.631929 1197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:12.615717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:35393 (local address 127.9.157.130:39529)
0811 17:22:12.616169 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:12.616200 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:12.616256 (+ 56us) server_negotiation.cc:407] Connection header received
0811 17:22:12.617014 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:12.617021 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:12.617395 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:12.617736 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:12.619252 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:12.620389 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:12.621889 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:12.622515 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:12.622743 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:12.626635 (+ 3892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:12.626646 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:12.626660 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:12.626702 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:12.628855 (+ 2153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:12.629543 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:12.629551 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:12.629556 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:12.629694 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:12.630089 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:12.630093 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:12.630095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:12.630587 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:12.630916 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:12.631344 (+ 428us) server_negotiation.cc:299] Negotiation successful
0811 17:22:12.631718 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":119,"threads_started":1}
I20260811 17:22:12.631949 1193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:12.615430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:39529 (local address 127.9.157.129:35393)
0811 17:22:12.615605 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:12.615627 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:12.615813 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:12.617813 (+ 2000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:12.617820 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:12.618443 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:22:12.619034 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:12.619054 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:12.620566 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:12.620570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:12.621744 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:12.621758 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:12.622663 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:12.622669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:12.622892 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:12.623802 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:22:12.623817 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:22:12.626387 (+ 2570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:22:12.629034 (+ 2647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:12.629042 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:12.629060 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:12.629378 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:12.629803 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:12.629807 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:12.629809 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:12.629980 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:12.630869 (+ 889us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:22:12.630873 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:22:12.630991 (+ 118us) client_negotiation.cc:769] Sending connection context
0811 17:22:12.631198 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 17:22:12.631718 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":316,"spinlock_wait_cycles":30464}
I20260811 17:22:12.632717 1200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:12.616034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:36517 (local address 127.9.157.131:40565)
0811 17:22:12.616420 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:12.616431 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:12.616463 (+ 32us) server_negotiation.cc:407] Connection header received
0811 17:22:12.617101 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:12.617107 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:12.617435 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:12.617743 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:12.619251 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:12.620398 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:12.621895 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:12.622502 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:12.622701 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:12.626388 (+ 3687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:12.626395 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:12.626414 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:12.626486 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:12.629140 (+ 2654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:12.629684 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:12.629690 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:12.629694 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:12.629824 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:12.630135 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:12.630140 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:12.630141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:12.630587 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:12.630740 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:12.631475 (+ 735us) server_negotiation.cc:299] Negotiation successful
0811 17:22:12.631810 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":110,"threads_started":1}
W20260811 17:22:12.635188 604 leader_election.cc:341] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ceb96fcae8fc40b7a0d2f73592090af9 (127.9.157.131:40565): Not found: Tablet not found: 2e99668915324cd9962e40a7e97ff179
W20260811 17:22:12.635396 604 leader_election.cc:341] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 08aa841a1f38462ea7ab7c8fe54d9de0 (127.9.157.130:39529): Not found: Tablet not found: 2e99668915324cd9962e40a7e97ff179
I20260811 17:22:12.635444 604 leader_election.cc:302] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6308b38e7b554a12861b5e97ab5c0aa3; no voters: 08aa841a1f38462ea7ab7c8fe54d9de0, ceb96fcae8fc40b7a0d2f73592090af9
I20260811 17:22:12.635785 1196 raft_consensus.cc:2726] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 17:22:12.788539 1202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:12.788026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.131:59113)
0811 17:22:12.788338 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:12.788423 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":87,"threads_started":1}
I20260811 17:22:13.440748 1205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:13.440283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.129:57165)
0811 17:22:13.440577 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:13.440643 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":90,"threads_started":1}
I20260811 17:22:13.567847 1206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:13.567248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.130:57027)
0811 17:22:13.567579 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:13.567705 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":157,"threads_started":1}
I20260811 17:22:13.790159 1207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:13.789578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.131:58473)
0811 17:22:13.789903 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:13.789974 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":89,"threads_started":1}
I20260811 17:22:14.442157 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:14.441675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.129:57189)
0811 17:22:14.441984 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:14.442049 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":97,"threads_started":1}
I20260811 17:22:14.551205 1209 raft_consensus.cc:482] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:22:14.551359 1209 raft_consensus.cc:504] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } }
I20260811 17:22:14.551903 1209 leader_election.cc:288] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 08aa841a1f38462ea7ab7c8fe54d9de0 (127.9.157.130:39529), ceb96fcae8fc40b7a0d2f73592090af9 (127.9.157.131:40565)
W20260811 17:22:14.552860 604 leader_election.cc:341] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ceb96fcae8fc40b7a0d2f73592090af9 (127.9.157.131:40565): Not found: Tablet not found: 2e99668915324cd9962e40a7e97ff179
W20260811 17:22:14.553030 604 leader_election.cc:341] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 08aa841a1f38462ea7ab7c8fe54d9de0 (127.9.157.130:39529): Not found: Tablet not found: 2e99668915324cd9962e40a7e97ff179
I20260811 17:22:14.553092 604 leader_election.cc:302] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6308b38e7b554a12861b5e97ab5c0aa3; no voters: 08aa841a1f38462ea7ab7c8fe54d9de0, ceb96fcae8fc40b7a0d2f73592090af9
I20260811 17:22:14.553293 1209 raft_consensus.cc:2726] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 17:22:14.569331 1212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:14.568803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.130:43007)
0811 17:22:14.569154 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:14.569222 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":138,"threads_started":1}
I20260811 17:22:14.791844 1214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:14.791208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.131:46261)
0811 17:22:14.791545 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:14.791621 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":104,"threads_started":1}
I20260811 17:22:15.443797 1215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:15.443217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.129:59665)
0811 17:22:15.443539 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:15.443611 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":87,"threads_started":1}
I20260811 17:22:15.571063 1216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:15.570326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.130:42831)
0811 17:22:15.570708 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:15.570924 (+ 216us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":148,"threads_started":1}
I20260811 17:22:15.793484 1217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:15.792913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.131:58671)
0811 17:22:15.793299 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:15.793372 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":125,"threads_started":1}
I20260811 17:22:16.445664 1218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:16.444992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.129:36675)
0811 17:22:16.445352 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:16.445421 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":85,"threads_started":1}
I20260811 17:22:16.572757 1221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:16.572225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.130:38921)
0811 17:22:16.572588 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:16.572653 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":102,"threads_started":1}
I20260811 17:22:16.602262 1223 raft_consensus.cc:482] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:22:16.602434 1223 raft_consensus.cc:504] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } }
I20260811 17:22:16.603001 1223 leader_election.cc:288] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 08aa841a1f38462ea7ab7c8fe54d9de0 (127.9.157.130:39529), ceb96fcae8fc40b7a0d2f73592090af9 (127.9.157.131:40565)
W20260811 17:22:16.604085 604 leader_election.cc:341] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ceb96fcae8fc40b7a0d2f73592090af9 (127.9.157.131:40565): Not found: Tablet not found: 2e99668915324cd9962e40a7e97ff179
W20260811 17:22:16.604290 604 leader_election.cc:341] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 08aa841a1f38462ea7ab7c8fe54d9de0 (127.9.157.130:39529): Not found: Tablet not found: 2e99668915324cd9962e40a7e97ff179
I20260811 17:22:16.604350 604 leader_election.cc:302] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6308b38e7b554a12861b5e97ab5c0aa3; no voters: 08aa841a1f38462ea7ab7c8fe54d9de0, ceb96fcae8fc40b7a0d2f73592090af9
I20260811 17:22:16.604578 1223 raft_consensus.cc:2726] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 17:22:16.794912 1224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:16.794475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.131:38337)
0811 17:22:16.794733 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:16.794790 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":82,"threads_started":1}
I20260811 17:22:17.447304 1227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:17.446739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.129:38911)
0811 17:22:17.447101 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:17.447182 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":131,"threads_started":1}
I20260811 17:22:17.574173 1230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:17.573656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.130:45205)
0811 17:22:17.573990 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:17.574060 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":86,"threads_started":1}
I20260811 17:22:17.796446 1231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:17.795965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.131:47441)
0811 17:22:17.796278 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:17.796344 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":109,"threads_started":1}
I20260811 17:22:18.448925 1232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:18.448448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.129:56019)
0811 17:22:18.448744 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:18.448815 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":88,"threads_started":1}
I20260811 17:22:18.575495 1233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:18.575001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.130:32985)
0811 17:22:18.575323 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:18.575387 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":87,"threads_started":1}
I20260811 17:22:18.798110 1234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:18.797533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.131:47847)
0811 17:22:18.797925 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:18.798002 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":159,"threads_started":1}
I20260811 17:22:19.450304 1236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:19.449886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.129:39055)
0811 17:22:19.450148 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:19.450203 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":82,"threads_started":1}
I20260811 17:22:19.576968 1239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:19.576437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.130:44435)
0811 17:22:19.576774 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:19.576845 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":104,"threads_started":1}
I20260811 17:22:19.799556 1240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:19.799056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.131:43329)
0811 17:22:19.799384 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:19.799449 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":91,"threads_started":1}
I20260811 17:22:20.451952 1246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:20.451341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.129:59709)
0811 17:22:20.451774 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:20.451845 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":106,"threads_started":1}
I20260811 17:22:20.578591 1247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:20.578010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.130:44719)
0811 17:22:20.578382 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:20.578461 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":107,"threads_started":1}
I20260811 17:22:20.801141 1249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:20.800596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.131:34933)
0811 17:22:20.800961 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:20.801033 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":115,"threads_started":1}
I20260811 17:22:21.118526 1250 raft_consensus.cc:482] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:22:21.118680 1250 raft_consensus.cc:504] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } }
I20260811 17:22:21.119254 1250 leader_election.cc:288] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 08aa841a1f38462ea7ab7c8fe54d9de0 (127.9.157.130:39529), ceb96fcae8fc40b7a0d2f73592090af9 (127.9.157.131:40565)
W20260811 17:22:21.120182 604 leader_election.cc:341] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 08aa841a1f38462ea7ab7c8fe54d9de0 (127.9.157.130:39529): Not found: Tablet not found: 2e99668915324cd9962e40a7e97ff179
W20260811 17:22:21.120322 604 leader_election.cc:341] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ceb96fcae8fc40b7a0d2f73592090af9 (127.9.157.131:40565): Not found: Tablet not found: 2e99668915324cd9962e40a7e97ff179
I20260811 17:22:21.120404 604 leader_election.cc:302] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6308b38e7b554a12861b5e97ab5c0aa3; no voters: 08aa841a1f38462ea7ab7c8fe54d9de0, ceb96fcae8fc40b7a0d2f73592090af9
I20260811 17:22:21.120595 1250 raft_consensus.cc:2726] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 17:22:21.453493 1251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:21.453014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.129:50299)
0811 17:22:21.453318 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:21.453386 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":86,"threads_started":1}
I20260811 17:22:21.580047 1256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:21.579546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.130:58277)
0811 17:22:21.579881 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:21.579945 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":163,"threads_started":1}
I20260811 17:22:21.802691 1257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:21.802136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.131:48707)
0811 17:22:21.802498 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:21.802577 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":86,"threads_started":1}
I20260811 17:22:22.454923 1259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:22.454385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.129:56755)
0811 17:22:22.454729 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:22.454815 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":103,"threads_started":1}
I20260811 17:22:22.581645 1260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:22.581117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.130:49495)
0811 17:22:22.581438 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:22.581509 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":87,"threads_started":1}
I20260811 17:22:22.804423 1262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:22.803940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.131:50079)
0811 17:22:22.804248 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:22.804316 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":128,"threads_started":1}
I20260811 17:22:23.456564 1263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:23.456044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.129:51367)
0811 17:22:23.456387 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:23.456456 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":106,"threads_started":1}
I20260811 17:22:23.583060 1264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:23.582581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.130:57397)
0811 17:22:23.582869 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:23.582956 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":103,"threads_started":1}
I20260811 17:22:23.805979 1265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:23.805422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.131:38523)
0811 17:22:23.805753 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:23.805827 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":115,"threads_started":1}
I20260811 17:22:24.457970 1266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:24.457430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.129:41609)
0811 17:22:24.457760 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:24.457861 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":91,"threads_started":1}
I20260811 17:22:24.584762 1271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:24.584098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.130:41183)
0811 17:22:24.584585 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:24.584654 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":94,"threads_started":1}
I20260811 17:22:24.807592 1272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:24.807056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.131:52401)
0811 17:22:24.807412 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:24.807482 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":105,"threads_started":1}
I20260811 17:22:24.917968 1273 raft_consensus.cc:482] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:22:24.918159 1273 raft_consensus.cc:504] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08aa841a1f38462ea7ab7c8fe54d9de0" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 39529 } } peers { permanent_uuid: "ceb96fcae8fc40b7a0d2f73592090af9" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 40565 } } peers { permanent_uuid: "6308b38e7b554a12861b5e97ab5c0aa3" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 32955 } }
I20260811 17:22:24.918694 1273 leader_election.cc:288] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 08aa841a1f38462ea7ab7c8fe54d9de0 (127.9.157.130:39529), ceb96fcae8fc40b7a0d2f73592090af9 (127.9.157.131:40565)
W20260811 17:22:24.919616 604 leader_election.cc:341] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ceb96fcae8fc40b7a0d2f73592090af9 (127.9.157.131:40565): Not found: Tablet not found: 2e99668915324cd9962e40a7e97ff179
W20260811 17:22:24.919853 604 leader_election.cc:341] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 08aa841a1f38462ea7ab7c8fe54d9de0 (127.9.157.130:39529): Not found: Tablet not found: 2e99668915324cd9962e40a7e97ff179
I20260811 17:22:24.919942 604 leader_election.cc:302] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6308b38e7b554a12861b5e97ab5c0aa3; no voters: 08aa841a1f38462ea7ab7c8fe54d9de0, ceb96fcae8fc40b7a0d2f73592090af9
I20260811 17:22:24.920148 1273 raft_consensus.cc:2726] T 2e99668915324cd9962e40a7e97ff179 P 6308b38e7b554a12861b5e97ab5c0aa3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 17:22:25.459347 1275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:25.458883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.129:41201)
0811 17:22:25.459173 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:25.459241 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":85,"threads_started":1}
I20260811 17:22:25.586239 1276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:25.585756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.130:55099)
0811 17:22:25.586070 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:25.586135 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":78,"threads_started":1}
I20260811 17:22:25.809213 1277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:25.808664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.131:53769)
0811 17:22:25.809026 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:25.809099 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":91,"threads_started":1}
I20260811 17:22:26.461120 1279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:26.460589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.129:57121)
0811 17:22:26.460949 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:26.461015 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":84,"threads_started":1}
I20260811 17:22:26.588091 1283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:26.587410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.130:42299)
0811 17:22:26.587912 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:26.587986 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":92,"threads_started":1}
I20260811 17:22:26.789491 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 528
I20260811 17:22:26.805276 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 779
I20260811 17:22:26.811872 1284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:26.810425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:44901 (local address 127.9.157.131:56783)
0811 17:22:26.811618 (+ 1193us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:26.811726 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:44901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1096,"thread_start_us":871,"threads_started":1}
I20260811 17:22:26.823992 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 992
2026-08-11T17:22:26Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (20236 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:22:26 dist-test-slave-mjjr krb5kdc[1287](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:22:26 dist-test-slave-mjjr krb5kdc[1287](info): set up 2 sockets
Aug 11 17:22:26 dist-test-slave-mjjr krb5kdc[1287](info): commencing operation
krb5kdc: starting...
W20260811 17:22:28.115536 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:22:28 dist-test-slave-mjjr krb5kdc[1287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:22:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:22:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:22:28.268345 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:43235
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:43235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:22:28.394908 1305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:22:28.395251 1305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:22:28.395294 1305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:22:28.399887 1305 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 17:22:28.399969 1305 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 17:22:28.399986 1305 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 17:22:28.400000 1305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:22:28.400023 1305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:22:28.400043 1305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:22:28.400100 1305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:22:28.400122 1305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:22:28.406198 1305 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:43235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:43235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:22:28.407536 1305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:22:28.408692 1305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:22:28.418510 1316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:28.418570 1314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:28.418576 1313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:28.418715 1305 server_base.cc:1034] running on GCE node
I20260811 17:22:28.419502 1305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:22:28.420811 1305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:22:28.422039 1305 hybrid_clock.cc:648] HybridClock initialized: now 1786468948422012 us; error 52 us; skew 500 ppm
Aug 11 17:22:28 dist-test-slave-mjjr krb5kdc[1287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468948, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:22:28.425141 1305 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:22:28.426131 1305 webserver.cc:460] Webserver started at http://127.9.157.190:42677/ using document root <none> and password file <none>
I20260811 17:22:28.426740 1305 fs_manager.cc:359] Metadata directory not provided
I20260811 17:22:28.426808 1305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:22:28.427006 1305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:22:28.429342 1305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "5841d04b55e64deab20fa9ac3cd50389"
format_stamp: "Formatted at 2026-08-11 17:22:28 on dist-test-slave-mjjr"
server_key: "c9477597b809516e48ece22892c49a0d"
server_key_iv: "8b139dffa22d611a384b01763ddfd88d"
server_key_version: "encryptionkey@0"
I20260811 17:22:28.430022 1305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "5841d04b55e64deab20fa9ac3cd50389"
format_stamp: "Formatted at 2026-08-11 17:22:28 on dist-test-slave-mjjr"
server_key: "c9477597b809516e48ece22892c49a0d"
server_key_iv: "8b139dffa22d611a384b01763ddfd88d"
server_key_version: "encryptionkey@0"
I20260811 17:22:28.434363 1305 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 17:22:28.437103 1328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:28.438560 1305 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:22:28.438701 1305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "5841d04b55e64deab20fa9ac3cd50389"
format_stamp: "Formatted at 2026-08-11 17:22:28 on dist-test-slave-mjjr"
server_key: "c9477597b809516e48ece22892c49a0d"
server_key_iv: "8b139dffa22d611a384b01763ddfd88d"
server_key_version: "encryptionkey@0"
I20260811 17:22:28.438819 1305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:22:28.458551 1305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:22:28.461863 1305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:22:28.462044 1305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:22:28.470000 1305 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:43235
I20260811 17:22:28.470335 1402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:43235 every 8 connection(s)
I20260811 17:22:28.471134 1305 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:22:28.474519 1404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:22:28.477694 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 1305
I20260811 17:22:28.477813 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:22:28.478135 9846 external_mini_cluster.cc:1428] Setting key e36d5fbd92237b4462c6c802b8eeb027
I20260811 17:22:28.481323 1404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389: Bootstrap starting.
I20260811 17:22:28.484342 1404 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389: Neither blocks nor log segments found. Creating new log.
I20260811 17:22:28.485325 1404 log.cc:824] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389: Log is configured to *not* fsync() on all Append() calls
Aug 11 17:22:28 dist-test-slave-mjjr krb5kdc[1287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:22:28.488127 1404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389: No bootstrap required, opened a new log
I20260811 17:22:28.491254 1404 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5841d04b55e64deab20fa9ac3cd50389" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43235 } }
I20260811 17:22:28.491551 1404 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:22:28.491614 1404 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5841d04b55e64deab20fa9ac3cd50389, State: Initialized, Role: FOLLOWER
I20260811 17:22:28.492213 1404 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5841d04b55e64deab20fa9ac3cd50389" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43235 } }
I20260811 17:22:28.492479 1404 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:22:28.492553 1404 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:22:28.492648 1404 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:22:28.493638 1404 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5841d04b55e64deab20fa9ac3cd50389" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43235 } }
I20260811 17:22:28.494011 1404 leader_election.cc:302] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5841d04b55e64deab20fa9ac3cd50389; no voters:
I20260811 17:22:28.494419 1404 leader_election.cc:288] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:22:28.494504 1412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:22:28.494691 1412 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [term 1 LEADER]: Becoming Leader. State: Replica: 5841d04b55e64deab20fa9ac3cd50389, State: Running, Role: LEADER
I20260811 17:22:28.494874 1410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:28.479765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47168 (local address 127.9.157.190:43235)
0811 17:22:28.480224 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:28.480240 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:28.480280 (+ 40us) server_negotiation.cc:407] Connection header received
0811 17:22:28.480971 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:28.480990 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:28.481356 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:28.481701 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:28.482680 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:28.483802 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:28.485030 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:28.485572 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:28.485696 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:28.488765 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:28.488770 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:28.488786 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:28.488817 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:28.491093 (+ 2276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:28.491610 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:28.491619 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:28.491627 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:28.491821 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:28.492192 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:28.492196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:28.492198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:28.492892 (+ 694us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:28.493086 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:28.493431 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 17:22:28.493775 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":146,"threads_started":1}
I20260811 17:22:28.495085 1412 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5841d04b55e64deab20fa9ac3cd50389" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43235 } }
I20260811 17:22:28.495831 1404 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:22:28.497154 1414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5841d04b55e64deab20fa9ac3cd50389" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5841d04b55e64deab20fa9ac3cd50389" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43235 } } }
I20260811 17:22:28.497171 1415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5841d04b55e64deab20fa9ac3cd50389. Latest consensus state: current_term: 1 leader_uuid: "5841d04b55e64deab20fa9ac3cd50389" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5841d04b55e64deab20fa9ac3cd50389" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43235 } } }
I20260811 17:22:28.497313 1414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [sys.catalog]: This master's current role is: LEADER
I20260811 17:22:28.497318 1415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [sys.catalog]: This master's current role is: LEADER
I20260811 17:22:28.497762 1426 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:22:28.501250 1426 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:22:28.507309 1426 catalog_manager.cc:1347] Generated new cluster ID: fbab6d78e9684dad889bfbc7f6c34d20
I20260811 17:22:28.507427 1426 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:22:28.520422 1426 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:22:28.521499 1426 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:22:28.530632 1426 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389: Generated new TSK 0
I20260811 17:22:28.531489 1426 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
I20260811 17:22:28.570410 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:0
--local_ip_for_outbound_sockets=127.9.157.129
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:43235
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:22:28.701555 1444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:22:28.701884 1444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:22:28.701931 1444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:22:28.706461 1444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:22:28.706553 1444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:22:28.706617 1444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:22:28.713085 1444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:43235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:22:28.714433 1444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:22:28.715713 1444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:22:28.726958 1451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:28.726958 1454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:28.726958 1450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:28.727301 1444 server_base.cc:1034] running on GCE node
I20260811 17:22:28.727880 1444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:22:28.728667 1444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:22:28.729923 1444 hybrid_clock.cc:648] HybridClock initialized: now 1786468948729899 us; error 37 us; skew 500 ppm
Aug 11 17:22:28 dist-test-slave-mjjr krb5kdc[1287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468948, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:22:28.733297 1444 init.cc:377] Logged in from keytab as kudu/127.9.157.129@KRBTEST.COM (short username kudu)
I20260811 17:22:28.734441 1444 webserver.cc:460] Webserver started at http://127.9.157.129:40263/ using document root <none> and password file <none>
I20260811 17:22:28.735121 1444 fs_manager.cc:359] Metadata directory not provided
I20260811 17:22:28.735193 1444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:22:28.735387 1444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:22:28.737808 1444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/data/instance:
uuid: "b46d01b7d14947f9931fdea29f0f8e3c"
format_stamp: "Formatted at 2026-08-11 17:22:28 on dist-test-slave-mjjr"
server_key: "631d8b2cf58a98dfe5d8d5987af0c263"
server_key_iv: "776d93aa1f25f09dab2f5478917eadf3"
server_key_version: "encryptionkey@0"
I20260811 17:22:28.738415 1444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance:
uuid: "b46d01b7d14947f9931fdea29f0f8e3c"
format_stamp: "Formatted at 2026-08-11 17:22:28 on dist-test-slave-mjjr"
server_key: "631d8b2cf58a98dfe5d8d5987af0c263"
server_key_iv: "776d93aa1f25f09dab2f5478917eadf3"
server_key_version: "encryptionkey@0"
I20260811 17:22:28.743100 1444 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:22:28.746399 1463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:28.748013 1444 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:22:28.748142 1444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "b46d01b7d14947f9931fdea29f0f8e3c"
format_stamp: "Formatted at 2026-08-11 17:22:28 on dist-test-slave-mjjr"
server_key: "631d8b2cf58a98dfe5d8d5987af0c263"
server_key_iv: "776d93aa1f25f09dab2f5478917eadf3"
server_key_version: "encryptionkey@0"
I20260811 17:22:28.748250 1444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:22:28.765273 1444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:22:28.769392 1444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:22:28.769595 1444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:22:28.770399 1444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:22:28.771363 1444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:22:28.771416 1444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:28.771466 1444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:22:28.771481 1444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:28.782286 1444 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:43025
I20260811 17:22:28.782336 1631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:43025 every 8 connection(s)
I20260811 17:22:28.783383 1444 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:22:28.790804 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 1444
I20260811 17:22:28.790972 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance
I20260811 17:22:28.791337 9846 external_mini_cluster.cc:1428] Setting key 4937a106dfa0b2f5cff2ffb250dae849
Aug 11 17:22:28 dist-test-slave-mjjr krb5kdc[1287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468948, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:22:28.801618 1410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:28.785846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:52335 (local address 127.9.157.190:43235)
0811 17:22:28.786045 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:28.786051 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:28.787189 (+ 1138us) server_negotiation.cc:407] Connection header received
0811 17:22:28.788876 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:28.788880 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:28.788936 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:28.789037 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:28.790634 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:28.791542 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:28.793267 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:28.793521 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:28.793647 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:28.796875 (+ 3228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:28.796879 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:28.796881 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:28.796913 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:28.798852 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:28.799780 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:28.799789 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:28.799791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:28.799890 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:28.800327 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:28.800331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:28.800333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:28.800559 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:28.800681 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:28.801290 (+ 609us) server_negotiation.cc:299] Negotiation successful
0811 17:22:28.801439 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 17:22:28.802585 1635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:28.786197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.129:52335)
0811 17:22:28.786990 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:28.787051 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:28.788519 (+ 1468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:28.789278 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:28.789287 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:28.789716 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:22:28.790328 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:28.790394 (+ 66us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:28.791763 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:28.791767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:28.793119 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:28.793133 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:28.793729 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:28.793735 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:28.793938 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:28.794664 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:22:28.794683 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:22:28.796617 (+ 1934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:22:28.799192 (+ 2575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:28.799200 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:28.799224 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:28.799610 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:28.800014 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:28.800018 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:28.800020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:28.800205 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:28.800682 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:22:28.800693 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:22:28.801014 (+ 321us) client_negotiation.cc:769] Sending connection context
0811 17:22:28.801233 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 17:22:28.801638 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":151,"threads_started":1}
WARNING: no policy specified for kudu/127.9.157.130@KRBTEST.COM; defaulting to no policy
I20260811 17:22:28.804091 1632 heartbeater.cc:347] Connected to a master server at 127.9.157.190:43235
I20260811 17:22:28.804436 1632 heartbeater.cc:464] Registering TS with master...
I20260811 17:22:28.805207 1632 heartbeater.cc:511] Master 127.9.157.190:43235 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.130@KRBTEST.COM" created.
I20260811 17:22:28.807367 1354 ts_manager.cc:194] Registered new tserver with Master: b46d01b7d14947f9931fdea29f0f8e3c (127.9.157.129:43025)
I20260811 17:22:28.808717 1354 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.129@KRBTEST.COM'} at 127.9.157.129:52335
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
I20260811 17:22:28.828038 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:0
--local_ip_for_outbound_sockets=127.9.157.130
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:43235
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:22:28.956188 1644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:22:28.956516 1644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:22:28.956604 1644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:22:28.961071 1644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:22:28.961164 1644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:22:28.961274 1644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:22:28.967401 1644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:43235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:22:28.968734 1644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:22:28.969871 1644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:22:28.980692 1644 server_base.cc:1034] running on GCE node
W20260811 17:22:28.980633 1652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:28.980619 1651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:28.980619 1654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:28.981350 1644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:22:28.982101 1644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:22:28.983322 1644 hybrid_clock.cc:648] HybridClock initialized: now 1786468948983301 us; error 32 us; skew 500 ppm
Aug 11 17:22:28 dist-test-slave-mjjr krb5kdc[1287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468948, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:22:28.986688 1644 init.cc:377] Logged in from keytab as kudu/127.9.157.130@KRBTEST.COM (short username kudu)
I20260811 17:22:28.987877 1644 webserver.cc:460] Webserver started at http://127.9.157.130:44369/ using document root <none> and password file <none>
I20260811 17:22:28.988495 1644 fs_manager.cc:359] Metadata directory not provided
I20260811 17:22:28.988561 1644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:22:28.988740 1644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:22:28.990882 1644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/data/instance:
uuid: "87116910e3734e87b64cd77012c35417"
format_stamp: "Formatted at 2026-08-11 17:22:28 on dist-test-slave-mjjr"
server_key: "661f32eccb2e703c8e387ba31498acd3"
server_key_iv: "084cc39eb3ba48895a592a345887e9fc"
server_key_version: "encryptionkey@0"
I20260811 17:22:28.991426 1644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance:
uuid: "87116910e3734e87b64cd77012c35417"
format_stamp: "Formatted at 2026-08-11 17:22:28 on dist-test-slave-mjjr"
server_key: "661f32eccb2e703c8e387ba31498acd3"
server_key_iv: "084cc39eb3ba48895a592a345887e9fc"
server_key_version: "encryptionkey@0"
I20260811 17:22:28.995424 1644 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 17:22:28.998754 1663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:29.000463 1644 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:22:29.000659 1644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "87116910e3734e87b64cd77012c35417"
format_stamp: "Formatted at 2026-08-11 17:22:28 on dist-test-slave-mjjr"
server_key: "661f32eccb2e703c8e387ba31498acd3"
server_key_iv: "084cc39eb3ba48895a592a345887e9fc"
server_key_version: "encryptionkey@0"
I20260811 17:22:29.000823 1644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:22:29.018216 1644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:22:29.021976 1644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:22:29.022236 1644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:22:29.022965 1644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:22:29.023991 1644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:22:29.024044 1644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:29.024117 1644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:22:29.024158 1644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:29.034790 1644 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:40213
I20260811 17:22:29.034869 1826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:40213 every 8 connection(s)
I20260811 17:22:29.035945 1644 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:22:29.037273 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 1644
I20260811 17:22:29.037417 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance
I20260811 17:22:29.037724 9846 external_mini_cluster.cc:1428] Setting key 4c3518c6e1045a16a41251893eb286f9
Aug 11 17:22:29 dist-test-slave-mjjr krb5kdc[1287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468948, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.157.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.131@KRBTEST.COM" created.
I20260811 17:22:29.052860 1410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:29.038279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:37125 (local address 127.9.157.190:43235)
0811 17:22:29.038440 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:29.038445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:29.039619 (+ 1174us) server_negotiation.cc:407] Connection header received
0811 17:22:29.041149 (+ 1530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:29.041155 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:29.041226 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:29.041329 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:29.042735 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.043312 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.044744 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.045008 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.045137 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:29.048387 (+ 3250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:29.048392 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:29.048395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:29.048440 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:29.050117 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:29.050777 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:29.050781 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:29.050782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:29.050869 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:29.051305 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:29.051308 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:29.051309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:29.051594 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:29.051789 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:29.052567 (+ 778us) server_negotiation.cc:299] Negotiation successful
0811 17:22:29.052722 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 17:22:29.053781 1831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:29.038819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.130:37125)
0811 17:22:29.039425 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:29.039486 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:29.040896 (+ 1410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:29.041510 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:29.041517 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:29.041938 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:22:29.042567 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:29.042586 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.043468 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.043473 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:29.044593 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:29.044606 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.045194 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.045198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:29.045403 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:29.046089 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:22:29.046117 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:22:29.048116 (+ 1999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:22:29.050274 (+ 2158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:29.050284 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:29.050307 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:29.050630 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:29.050981 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:29.050983 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:29.050984 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:29.051143 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:29.051789 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:22:29.051795 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:22:29.052196 (+ 401us) client_negotiation.cc:769] Sending connection context
0811 17:22:29.052440 (+ 244us) client_negotiation.cc:240] Negotiation successful
0811 17:22:29.052824 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":162,"threads_started":1}
I20260811 17:22:29.055331 1829 heartbeater.cc:347] Connected to a master server at 127.9.157.190:43235
I20260811 17:22:29.055693 1829 heartbeater.cc:464] Registering TS with master...
I20260811 17:22:29.056365 1829 heartbeater.cc:511] Master 127.9.157.190:43235 requested a full tablet report, sending...
I20260811 17:22:29.057655 1354 ts_manager.cc:194] Registered new tserver with Master: 87116910e3734e87b64cd77012c35417 (127.9.157.130:40213)
I20260811 17:22:29.058331 1354 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.130@KRBTEST.COM'} at 127.9.157.130:37125
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
I20260811 17:22:29.071566 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:0
--local_ip_for_outbound_sockets=127.9.157.131
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:43235
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:22:29.201520 1839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:22:29.201812 1839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:22:29.201858 1839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:22:29.206415 1839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:22:29.206535 1839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:22:29.206601 1839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:22:29.212598 1839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:43235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:22:29.213856 1839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:22:29.215049 1839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:22:29.226054 1850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:29.226060 1845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:29.226055 1847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:29.226730 1839 server_base.cc:1034] running on GCE node
I20260811 17:22:29.227236 1839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:22:29.228050 1839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:22:29.229223 1839 hybrid_clock.cc:648] HybridClock initialized: now 1786468949229182 us; error 51 us; skew 500 ppm
Aug 11 17:22:29 dist-test-slave-mjjr krb5kdc[1287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468949, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:22:29.232303 1839 init.cc:377] Logged in from keytab as kudu/127.9.157.131@KRBTEST.COM (short username kudu)
I20260811 17:22:29.233336 1839 webserver.cc:460] Webserver started at http://127.9.157.131:41123/ using document root <none> and password file <none>
I20260811 17:22:29.233997 1839 fs_manager.cc:359] Metadata directory not provided
I20260811 17:22:29.234104 1839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:22:29.234413 1839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:22:29.237051 1839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/data/instance:
uuid: "f5ca2cdc8d61405080d925fa399fc98b"
format_stamp: "Formatted at 2026-08-11 17:22:29 on dist-test-slave-mjjr"
server_key: "50746328f5c37fd65a3fb7a4898d0010"
server_key_iv: "68c9a7a7ec8b1cee68c5c4e33c363df2"
server_key_version: "encryptionkey@0"
I20260811 17:22:29.237758 1839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance:
uuid: "f5ca2cdc8d61405080d925fa399fc98b"
format_stamp: "Formatted at 2026-08-11 17:22:29 on dist-test-slave-mjjr"
server_key: "50746328f5c37fd65a3fb7a4898d0010"
server_key_iv: "68c9a7a7ec8b1cee68c5c4e33c363df2"
server_key_version: "encryptionkey@0"
I20260811 17:22:29.242161 1839 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:22:29.245242 1858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:29.246857 1839 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:22:29.247059 1839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "f5ca2cdc8d61405080d925fa399fc98b"
format_stamp: "Formatted at 2026-08-11 17:22:29 on dist-test-slave-mjjr"
server_key: "50746328f5c37fd65a3fb7a4898d0010"
server_key_iv: "68c9a7a7ec8b1cee68c5c4e33c363df2"
server_key_version: "encryptionkey@0"
I20260811 17:22:29.247218 1839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:22:29.267052 1839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:22:29.270736 1839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:22:29.270977 1839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:22:29.271713 1839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:22:29.272650 1839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:22:29.272725 1839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:29.272800 1839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:22:29.272859 1839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:29.283192 1839 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:36241
I20260811 17:22:29.283232 2033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:36241 every 8 connection(s)
I20260811 17:22:29.284307 1839 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:22:29.291328 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 1839
I20260811 17:22:29.291456 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance
I20260811 17:22:29.291785 9846 external_mini_cluster.cc:1428] Setting key 7a5e4902dfe955fc70159d8ea3a72a3a
Aug 11 17:22:29 dist-test-slave-mjjr krb5kdc[1287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468949, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:22:29.299213 1410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:29.286462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:56389 (local address 127.9.157.190:43235)
0811 17:22:29.286657 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:29.286661 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:29.287567 (+ 906us) server_negotiation.cc:407] Connection header received
0811 17:22:29.289001 (+ 1434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:29.289003 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:29.289053 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:29.289131 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:29.290400 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.290957 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.292410 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.292714 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.292831 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:29.295505 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:29.295508 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:29.295511 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:29.295536 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:29.296926 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:29.297606 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:29.297609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:29.297610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:29.297690 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:29.298070 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:29.298073 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:29.298075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:29.298354 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:29.298436 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:29.298971 (+ 535us) server_negotiation.cc:299] Negotiation successful
0811 17:22:29.299085 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 17:22:29.300223 2037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:29.286866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.131:56389)
0811 17:22:29.287387 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:29.287435 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:29.288781 (+ 1346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:29.289318 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:29.289325 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:29.289710 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:22:29.290244 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:29.290258 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.291081 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.291083 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:29.292262 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:29.292273 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.292914 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.292922 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:29.293099 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:29.293726 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:22:29.293748 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:22:29.295299 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:22:29.297066 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:29.297071 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:29.297085 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:29.297515 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:29.297799 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:29.297801 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:29.297803 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:29.297945 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:29.298451 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:22:29.298457 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:22:29.298721 (+ 264us) client_negotiation.cc:769] Sending connection context
0811 17:22:29.298909 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 17:22:29.299186 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":127,"threads_started":1}
I20260811 17:22:29.301453 2035 heartbeater.cc:347] Connected to a master server at 127.9.157.190:43235
I20260811 17:22:29.301689 2035 heartbeater.cc:464] Registering TS with master...
I20260811 17:22:29.302174 2035 heartbeater.cc:511] Master 127.9.157.190:43235 requested a full tablet report, sending...
I20260811 17:22:29.303231 1354 ts_manager.cc:194] Registered new tserver with Master: f5ca2cdc8d61405080d925fa399fc98b (127.9.157.131:36241)
I20260811 17:22:29.303918 1354 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.131@KRBTEST.COM'} at 127.9.157.131:56389
I20260811 17:22:29.306237 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:22:29.315801 1410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:29.308432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47180 (local address 127.9.157.190:43235)
0811 17:22:29.308570 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:29.308574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:29.308667 (+ 93us) server_negotiation.cc:407] Connection header received
0811 17:22:29.308766 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:29.308768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:29.308808 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:29.308884 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:29.309611 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.310168 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.311016 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.311229 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.311315 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:29.312804 (+ 1489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:29.312808 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:29.312810 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:29.312834 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:29.314130 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:29.314537 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:29.314540 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:29.314541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:29.314616 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:29.314912 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:29.314914 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:29.314915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:29.315144 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:29.315251 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:29.315472 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 17:22:29.315580 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 17:22:29.319392 1354 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:22:29.322660 1354 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:22:29.339658 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:29.333293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52052 (local address 127.9.157.130:40213)
0811 17:22:29.334050 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:29.334067 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:29.334098 (+ 31us) server_negotiation.cc:407] Connection header received
0811 17:22:29.334303 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:29.334311 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:29.334552 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:29.334729 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:22:29.335076 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.336356 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.338250 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.338850 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.338987 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:29.339160 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:29.339350 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 17:22:29.339457 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":156,"threads_started":1}
I20260811 17:22:29.339630 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:29.332994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:40213 (local address 127.0.0.1:52052)
0811 17:22:29.333869 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:29.333931 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:29.334068 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:29.334744 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:29.334753 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:29.334781 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:22:29.334940 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:29.334947 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.336527 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.336532 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:29.338100 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:29.338111 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.339003 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.339006 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:29.339116 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:29.339155 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 17:22:29.339311 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 17:22:29.339456 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"mutex_wait_us":199,"thread_start_us":116,"threads_started":1}
I20260811 17:22:29.339905 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:29.332989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:43025 (local address 127.0.0.1:55812)
0811 17:22:29.334450 (+ 1461us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:29.334463 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:29.334636 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:29.335367 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:29.335370 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:29.335382 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:22:29.335498 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:29.335503 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.336567 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.336570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:29.338048 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:29.338060 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.339286 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.339290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:29.339493 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:29.339570 (+ 77us) client_negotiation.cc:769] Sending connection context
0811 17:22:29.339667 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 17:22:29.339766 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1384,"mutex_wait_us":142,"thread_start_us":142,"threads_started":1}
I20260811 17:22:29.340312 2049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:29.333694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:36241 (local address 127.0.0.1:35678)
0811 17:22:29.334658 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:29.334673 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:29.334764 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:29.335302 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:29.335308 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:29.335331 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:22:29.335482 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:29.335490 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.336904 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.336908 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:29.337935 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:29.337946 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.339286 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.339290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:29.339534 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:29.339570 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 17:22:29.339677 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 17:22:29.340190 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":876,"mutex_wait_us":568,"thread_start_us":97,"threads_started":1}
I20260811 17:22:29.340436 2051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:29.333090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55812 (local address 127.9.157.129:43025)
0811 17:22:29.334293 (+ 1203us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:29.334301 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:29.334555 (+ 254us) server_negotiation.cc:407] Connection header received
0811 17:22:29.334821 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:29.334826 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:29.335015 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:29.335179 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:22:29.335632 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.336405 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.338196 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.338969 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.339064 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:29.339210 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:29.340166 (+ 956us) server_negotiation.cc:299] Negotiation successful
0811 17:22:29.340281 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1097,"thread_start_us":94,"threads_started":1}
I20260811 17:22:29.340533 2057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:29.333988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35678 (local address 127.9.157.131:36241)
0811 17:22:29.334502 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:29.334516 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:29.334868 (+ 352us) server_negotiation.cc:407] Connection header received
0811 17:22:29.334961 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:29.334964 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:29.335165 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:29.335320 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:22:29.335614 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.336770 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.338079 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.338947 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.339065 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:29.339210 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:29.339765 (+ 555us) server_negotiation.cc:299] Negotiation successful
0811 17:22:29.339860 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":104,"threads_started":1}
I20260811 17:22:29.343724 1733 tablet_service.cc:1467] Processing CreateTablet for tablet 4775c95a1a524e0ab50c8775a233a16d (DEFAULT_TABLE table=test-table [id=c17ebc5f62dc4217bdac5c7f37701e2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:22:29.343931 1533 tablet_service.cc:1467] Processing CreateTablet for tablet 4775c95a1a524e0ab50c8775a233a16d (DEFAULT_TABLE table=test-table [id=c17ebc5f62dc4217bdac5c7f37701e2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:22:29.344913 1733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4775c95a1a524e0ab50c8775a233a16d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:22:29.344535 1940 tablet_service.cc:1467] Processing CreateTablet for tablet 4775c95a1a524e0ab50c8775a233a16d (DEFAULT_TABLE table=test-table [id=c17ebc5f62dc4217bdac5c7f37701e2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:22:29.345031 1533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4775c95a1a524e0ab50c8775a233a16d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:22:29.345534 1940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4775c95a1a524e0ab50c8775a233a16d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:22:29.352118 2058 tablet_bootstrap.cc:492] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417: Bootstrap starting.
I20260811 17:22:29.353791 2061 tablet_bootstrap.cc:492] T 4775c95a1a524e0ab50c8775a233a16d P f5ca2cdc8d61405080d925fa399fc98b: Bootstrap starting.
I20260811 17:22:29.355463 2058 tablet_bootstrap.cc:654] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417: Neither blocks nor log segments found. Creating new log.
I20260811 17:22:29.355759 2061 tablet_bootstrap.cc:654] T 4775c95a1a524e0ab50c8775a233a16d P f5ca2cdc8d61405080d925fa399fc98b: Neither blocks nor log segments found. Creating new log.
I20260811 17:22:29.356102 2060 tablet_bootstrap.cc:492] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c: Bootstrap starting.
I20260811 17:22:29.356482 2058 log.cc:824] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417: Log is configured to *not* fsync() on all Append() calls
I20260811 17:22:29.357092 2061 log.cc:824] T 4775c95a1a524e0ab50c8775a233a16d P f5ca2cdc8d61405080d925fa399fc98b: Log is configured to *not* fsync() on all Append() calls
I20260811 17:22:29.358487 2060 tablet_bootstrap.cc:654] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c: Neither blocks nor log segments found. Creating new log.
I20260811 17:22:29.358832 2058 tablet_bootstrap.cc:492] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417: No bootstrap required, opened a new log
I20260811 17:22:29.359052 2058 ts_tablet_manager.cc:1397] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 17:22:29.359346 2061 tablet_bootstrap.cc:492] T 4775c95a1a524e0ab50c8775a233a16d P f5ca2cdc8d61405080d925fa399fc98b: No bootstrap required, opened a new log
I20260811 17:22:29.359346 2060 log.cc:824] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c: Log is configured to *not* fsync() on all Append() calls
I20260811 17:22:29.359530 2061 ts_tablet_manager.cc:1397] T 4775c95a1a524e0ab50c8775a233a16d P f5ca2cdc8d61405080d925fa399fc98b: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 17:22:29.361974 2060 tablet_bootstrap.cc:492] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c: No bootstrap required, opened a new log
I20260811 17:22:29.362182 2060 ts_tablet_manager.cc:1397] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 17:22:29.362721 2061 raft_consensus.cc:348] T 4775c95a1a524e0ab50c8775a233a16d P f5ca2cdc8d61405080d925fa399fc98b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } } peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } }
I20260811 17:22:29.362962 2061 raft_consensus.cc:374] T 4775c95a1a524e0ab50c8775a233a16d P f5ca2cdc8d61405080d925fa399fc98b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:22:29.362996 2061 raft_consensus.cc:729] T 4775c95a1a524e0ab50c8775a233a16d P f5ca2cdc8d61405080d925fa399fc98b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ca2cdc8d61405080d925fa399fc98b, State: Initialized, Role: FOLLOWER
I20260811 17:22:29.363090 2058 raft_consensus.cc:348] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } } peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } }
I20260811 17:22:29.363386 2058 raft_consensus.cc:374] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:22:29.363437 2058 raft_consensus.cc:729] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87116910e3734e87b64cd77012c35417, State: Initialized, Role: FOLLOWER
I20260811 17:22:29.363497 2061 consensus_queue.cc:260] T 4775c95a1a524e0ab50c8775a233a16d P f5ca2cdc8d61405080d925fa399fc98b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } } peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } }
I20260811 17:22:29.363920 2058 consensus_queue.cc:260] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } } peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } }
I20260811 17:22:29.364395 2035 heartbeater.cc:503] Master 127.9.157.190:43235 was elected leader, sending a full tablet report...
I20260811 17:22:29.364727 2061 ts_tablet_manager.cc:1428] T 4775c95a1a524e0ab50c8775a233a16d P f5ca2cdc8d61405080d925fa399fc98b: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 17:22:29.364977 2058 ts_tablet_manager.cc:1428] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 17:22:29.365376 1829 heartbeater.cc:503] Master 127.9.157.190:43235 was elected leader, sending a full tablet report...
I20260811 17:22:29.367007 2060 raft_consensus.cc:348] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } } peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } }
I20260811 17:22:29.367316 2060 raft_consensus.cc:374] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:22:29.367410 2060 raft_consensus.cc:729] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b46d01b7d14947f9931fdea29f0f8e3c, State: Initialized, Role: FOLLOWER
I20260811 17:22:29.367944 2060 consensus_queue.cc:260] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } } peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } }
I20260811 17:22:29.368729 1632 heartbeater.cc:503] Master 127.9.157.190:43235 was elected leader, sending a full tablet report...
I20260811 17:22:29.369149 2060 ts_tablet_manager.cc:1428] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c: Time spent starting tablet: real 0.007s user 0.000s sys 0.006s
I20260811 17:22:29.453539 2067 raft_consensus.cc:482] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:22:29.453776 2067 raft_consensus.cc:504] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } } peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } }
I20260811 17:22:29.455117 2067 leader_election.cc:288] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b46d01b7d14947f9931fdea29f0f8e3c (127.9.157.129:43025), f5ca2cdc8d61405080d925fa399fc98b (127.9.157.131:36241)
I20260811 17:22:29.459488 2069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:29.455488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:36241 (local address 127.9.157.130:34241)
0811 17:22:29.456142 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:29.456159 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:29.456265 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:29.456558 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:29.456561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:29.456581 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:22:29.456705 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:29.456712 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.457475 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.457477 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:29.458261 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:29.458269 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.458981 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.458985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:29.459093 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:29.459119 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 17:22:29.459221 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 17:22:29.459309 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":183,"thread_start_us":172,"threads_started":1}
I20260811 17:22:29.459476 2057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:29.455610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:34241 (local address 127.9.157.131:36241)
0811 17:22:29.456075 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:29.456079 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:29.456269 (+ 190us) server_negotiation.cc:407] Connection header received
0811 17:22:29.456357 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:29.456360 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:29.456423 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:29.456515 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:22:29.456820 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.457347 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.458434 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.458838 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.458986 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:29.459049 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:29.459247 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 17:22:29.459328 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383}
I20260811 17:22:29.459614 2051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:29.455747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:36961 (local address 127.9.157.129:43025)
0811 17:22:29.455933 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:29.455938 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:29.455957 (+ 19us) server_negotiation.cc:407] Connection header received
0811 17:22:29.456023 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:29.456026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:29.456077 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:29.456184 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:22:29.456597 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.457207 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.458395 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.458839 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.458988 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:29.459047 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:29.459388 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 17:22:29.459485 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 17:22:29.459951 1831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:29.455488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:43025 (local address 127.9.157.130:36961)
0811 17:22:29.455612 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:29.455628 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:29.455794 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:29.456269 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:29.456272 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:29.456299 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:22:29.456448 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:29.456459 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.457336 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.457339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:29.458228 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:29.458239 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.459004 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.459008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:29.459141 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:29.459163 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 17:22:29.459237 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 17:22:29.459311 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 17:22:29.460711 1560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4775c95a1a524e0ab50c8775a233a16d" candidate_uuid: "87116910e3734e87b64cd77012c35417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" is_pre_election: true
I20260811 17:22:29.460711 1966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4775c95a1a524e0ab50c8775a233a16d" candidate_uuid: "87116910e3734e87b64cd77012c35417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ca2cdc8d61405080d925fa399fc98b" is_pre_election: true
I20260811 17:22:29.461077 1966 raft_consensus.cc:2454] T 4775c95a1a524e0ab50c8775a233a16d P f5ca2cdc8d61405080d925fa399fc98b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87116910e3734e87b64cd77012c35417 in term 0.
I20260811 17:22:29.461077 1560 raft_consensus.cc:2454] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87116910e3734e87b64cd77012c35417 in term 0.
I20260811 17:22:29.461647 1667 leader_election.cc:302] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87116910e3734e87b64cd77012c35417, f5ca2cdc8d61405080d925fa399fc98b; no voters:
I20260811 17:22:29.461992 2067 raft_consensus.cc:2781] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:22:29.462088 2067 raft_consensus.cc:482] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:22:29.462136 2067 raft_consensus.cc:3037] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:22:29.463231 2067 raft_consensus.cc:504] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } } peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } }
I20260811 17:22:29.463718 2067 leader_election.cc:288] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417 [CANDIDATE]: Term 1 election: Requested vote from peers b46d01b7d14947f9931fdea29f0f8e3c (127.9.157.129:43025), f5ca2cdc8d61405080d925fa399fc98b (127.9.157.131:36241)
I20260811 17:22:29.464154 1966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4775c95a1a524e0ab50c8775a233a16d" candidate_uuid: "87116910e3734e87b64cd77012c35417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ca2cdc8d61405080d925fa399fc98b"
I20260811 17:22:29.464174 1560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4775c95a1a524e0ab50c8775a233a16d" candidate_uuid: "87116910e3734e87b64cd77012c35417" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b46d01b7d14947f9931fdea29f0f8e3c"
I20260811 17:22:29.464308 1966 raft_consensus.cc:3037] T 4775c95a1a524e0ab50c8775a233a16d P f5ca2cdc8d61405080d925fa399fc98b [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:22:29.464347 1560 raft_consensus.cc:3037] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:22:29.465519 1966 raft_consensus.cc:2454] T 4775c95a1a524e0ab50c8775a233a16d P f5ca2cdc8d61405080d925fa399fc98b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87116910e3734e87b64cd77012c35417 in term 1.
I20260811 17:22:29.465684 1560 raft_consensus.cc:2454] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87116910e3734e87b64cd77012c35417 in term 1.
I20260811 17:22:29.465904 1667 leader_election.cc:302] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87116910e3734e87b64cd77012c35417, f5ca2cdc8d61405080d925fa399fc98b; no voters:
I20260811 17:22:29.466116 2067 raft_consensus.cc:2781] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:22:29.466354 2067 raft_consensus.cc:686] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417 [term 1 LEADER]: Becoming Leader. State: Replica: 87116910e3734e87b64cd77012c35417, State: Running, Role: LEADER
I20260811 17:22:29.466804 2067 consensus_queue.cc:237] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } } peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } }
I20260811 17:22:29.470404 1354 catalog_manager.cc:5495] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417 reported cstate change: term changed from 0 to 1, leader changed from <none> to 87116910e3734e87b64cd77012c35417 (127.9.157.130). New cstate: current_term: 1 leader_uuid: "87116910e3734e87b64cd77012c35417" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } health_report { overall_health: UNKNOWN } } }
I20260811 17:22:29.499781 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:29.495611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52066 (local address 127.9.157.130:40213)
0811 17:22:29.495812 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:29.495816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:29.495831 (+ 15us) server_negotiation.cc:407] Connection header received
0811 17:22:29.495917 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:29.495919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:29.495982 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:29.496078 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:22:29.496320 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.496970 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.498078 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:29.498369 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:29.498469 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:29.498811 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:22:29.499292 (+ 481us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:22:29.499401 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:29.499495 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 17:22:29.499557 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 17:22:29.506553 1966 raft_consensus.cc:1261] T 4775c95a1a524e0ab50c8775a233a16d P f5ca2cdc8d61405080d925fa399fc98b [term 1 FOLLOWER]: Refusing update from remote peer 87116910e3734e87b64cd77012c35417: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:22:29.506556 1560 raft_consensus.cc:1261] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c [term 1 FOLLOWER]: Refusing update from remote peer 87116910e3734e87b64cd77012c35417: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:22:29.507477 2070 consensus_queue.cc:1035] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:22:29.507766 2067 consensus_queue.cc:1035] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:22:29.517612 2077 mvcc.cc:205] Tried to move back new op lower bound from 7317376817169719296 to 7317376817019428864. Current Snapshot: MvccSnapshot[applied={T|T < 7317376817169719296}]
I20260811 17:22:29.517835 2079 mvcc.cc:205] Tried to move back new op lower bound from 7317376817169719296 to 7317376817019428864. Current Snapshot: MvccSnapshot[applied={T|T < 7317376817169719296}]
I20260811 17:22:29.528264 1352 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:22:29.528465 1352 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:22:29.531510 1352 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389: Sending DeleteTablet for 3 replicas of tablet 4775c95a1a524e0ab50c8775a233a16d
I20260811 17:22:29.532601 1733 tablet_service.cc:1514] Processing DeleteTablet for tablet 4775c95a1a524e0ab50c8775a233a16d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:22:29 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:52052
I20260811 17:22:29.532629 1533 tablet_service.cc:1514] Processing DeleteTablet for tablet 4775c95a1a524e0ab50c8775a233a16d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:22:29 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:55812
I20260811 17:22:29.533275 2090 tablet_replica.cc:330] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417: stopping tablet replica
I20260811 17:22:29.533275 2091 tablet_replica.cc:330] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c: stopping tablet replica
I20260811 17:22:29.533689 2091 raft_consensus.cc:2229] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:22:29.533689 2090 raft_consensus.cc:2229] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417 [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:22:29.533710 1940 tablet_service.cc:1514] Processing DeleteTablet for tablet 4775c95a1a524e0ab50c8775a233a16d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:22:29 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:35678
I20260811 17:22:29.534157 2090 raft_consensus.cc:2258] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:22:29.534199 2091 raft_consensus.cc:2258] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:22:29.534478 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 1444
I20260811 17:22:29.534628 2092 tablet_replica.cc:330] T 4775c95a1a524e0ab50c8775a233a16d P f5ca2cdc8d61405080d925fa399fc98b: stopping tablet replica
I20260811 17:22:29.534924 2092 raft_consensus.cc:2229] T 4775c95a1a524e0ab50c8775a233a16d P f5ca2cdc8d61405080d925fa399fc98b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:22:29.535274 2092 raft_consensus.cc:2258] T 4775c95a1a524e0ab50c8775a233a16d P f5ca2cdc8d61405080d925fa399fc98b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:22:29.539469 2092 ts_tablet_manager.cc:1905] T 4775c95a1a524e0ab50c8775a233a16d P f5ca2cdc8d61405080d925fa399fc98b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:22:29.544129 2090 ts_tablet_manager.cc:1905] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:22:29.544445 2092 ts_tablet_manager.cc:1918] T 4775c95a1a524e0ab50c8775a233a16d P f5ca2cdc8d61405080d925fa399fc98b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:22:29.544581 2092 log.cc:1196] T 4775c95a1a524e0ab50c8775a233a16d P f5ca2cdc8d61405080d925fa399fc98b: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/wal/wals/4775c95a1a524e0ab50c8775a233a16d
I20260811 17:22:29.544981 2092 ts_tablet_manager.cc:1939] T 4775c95a1a524e0ab50c8775a233a16d P f5ca2cdc8d61405080d925fa399fc98b: Deleting consensus metadata
I20260811 17:22:29.546681 1332 catalog_manager.cc:4841] TS f5ca2cdc8d61405080d925fa399fc98b (127.9.157.131:36241): tablet 4775c95a1a524e0ab50c8775a233a16d (table test-table [id=c17ebc5f62dc4217bdac5c7f37701e2e]) successfully deleted
I20260811 17:22:29.547286 2090 ts_tablet_manager.cc:1918] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:22:29.547420 2090 log.cc:1196] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/wal/wals/4775c95a1a524e0ab50c8775a233a16d
I20260811 17:22:29.547804 2090 ts_tablet_manager.cc:1939] T 4775c95a1a524e0ab50c8775a233a16d P 87116910e3734e87b64cd77012c35417: Deleting consensus metadata
I20260811 17:22:29.549274 1329 catalog_manager.cc:4841] TS 87116910e3734e87b64cd77012c35417 (127.9.157.130:40213): tablet 4775c95a1a524e0ab50c8775a233a16d (table test-table [id=c17ebc5f62dc4217bdac5c7f37701e2e]) successfully deleted
W20260811 17:22:29.551523 1334 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.157.129:43025 (error 108)
I20260811 17:22:29.553128 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 1644
I20260811 17:22:29.568662 2056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:29.568075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:43025 (local address 127.0.0.1:55826)
0811 17:22:29.568405 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:29.568514 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.129:43025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209}
W20260811 17:22:29.571461 1334 catalog_manager.cc:4584] TS b46d01b7d14947f9931fdea29f0f8e3c (127.9.157.129:43025): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4775c95a1a524e0ab50c8775a233a16d: Network error: Client connection negotiation failed: client connection to 127.9.157.129:43025: connect: Connection refused (error 111)
I20260811 17:22:29.578519 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 1839
I20260811 17:22:29.599116 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 1305
I20260811 17:22:49.621129 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:43235
--webserver_interface=127.9.157.190
--webserver_port=42677
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:43235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:22:49.744261 2097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:22:49.744575 2097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:22:49.744618 2097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:22:49.748826 2097 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 17:22:49.748908 2097 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 17:22:49.748925 2097 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 17:22:49.748939 2097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:22:49.748960 2097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:22:49.748978 2097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:22:49.748993 2097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:22:49.749009 2097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:22:49.754509 2097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:43235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:43235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=42677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:22:49.755810 2097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:22:49.756909 2097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:22:49.772544 2107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:49.772514 2106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:49.772500 2110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:49.772595 2097 server_base.cc:1034] running on GCE node
I20260811 17:22:49.773525 2097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:22:49.774865 2097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:22:49.776062 2097 hybrid_clock.cc:648] HybridClock initialized: now 1786468969776031 us; error 55 us; skew 500 ppm
Aug 11 17:22:49 dist-test-slave-mjjr krb5kdc[1287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468969, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:22:49.779193 2097 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:22:49.780272 2097 webserver.cc:460] Webserver started at http://127.9.157.190:42677/ using document root <none> and password file <none>
I20260811 17:22:49.780979 2097 fs_manager.cc:359] Metadata directory not provided
I20260811 17:22:49.781100 2097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:22:49.785501 2097 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 17:22:49.788130 2119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:49.789570 2097 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:22:49.789737 2097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "5841d04b55e64deab20fa9ac3cd50389"
format_stamp: "Formatted at 2026-08-11 17:22:28 on dist-test-slave-mjjr"
server_key: "c9477597b809516e48ece22892c49a0d"
server_key_iv: "8b139dffa22d611a384b01763ddfd88d"
server_key_version: "encryptionkey@0"
I20260811 17:22:49.790428 2097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:22:49.814950 2097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:22:49.818832 2097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:22:49.819032 2097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:22:49.827128 2097 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:43235
I20260811 17:22:49.827487 2190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:43235 every 8 connection(s)
I20260811 17:22:49.828428 2097 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:22:49.832000 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 2097
I20260811 17:22:49.832563 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:43025
--local_ip_for_outbound_sockets=127.9.157.129
--tserver_master_addrs=127.9.157.190:43235
--webserver_port=40263
--webserver_interface=127.9.157.129
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:22:49.833973 2191 sys_catalog.cc:263] Verifying existing consensus state
I20260811 17:22:49.836443 2191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389: Bootstrap starting.
I20260811 17:22:49.859041 2191 log.cc:824] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389: Log is configured to *not* fsync() on all Append() calls
I20260811 17:22:49.871181 2191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 17:22:49.871758 2191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389: Bootstrap complete.
I20260811 17:22:49.877228 2191 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5841d04b55e64deab20fa9ac3cd50389" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43235 } }
I20260811 17:22:49.877696 2191 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5841d04b55e64deab20fa9ac3cd50389, State: Initialized, Role: FOLLOWER
I20260811 17:22:49.878273 2191 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5841d04b55e64deab20fa9ac3cd50389" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43235 } }
I20260811 17:22:49.878434 2191 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:22:49.878532 2191 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:22:49.878654 2191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [term 1 FOLLOWER]: Advancing to term 2
I20260811 17:22:49.880900 2191 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5841d04b55e64deab20fa9ac3cd50389" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43235 } }
I20260811 17:22:49.881266 2191 leader_election.cc:302] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5841d04b55e64deab20fa9ac3cd50389; no voters:
I20260811 17:22:49.881634 2191 leader_election.cc:288] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 17:22:49.881775 2199 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 17:22:49.882057 2199 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [term 2 LEADER]: Becoming Leader. State: Replica: 5841d04b55e64deab20fa9ac3cd50389, State: Running, Role: LEADER
I20260811 17:22:49.882411 2199 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5841d04b55e64deab20fa9ac3cd50389" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43235 } }
I20260811 17:22:49.882964 2191 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:22:49.883514 2199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5841d04b55e64deab20fa9ac3cd50389. Latest consensus state: current_term: 2 leader_uuid: "5841d04b55e64deab20fa9ac3cd50389" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5841d04b55e64deab20fa9ac3cd50389" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43235 } } }
I20260811 17:22:49.883632 2199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [sys.catalog]: This master's current role is: LEADER
I20260811 17:22:49.884152 2201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5841d04b55e64deab20fa9ac3cd50389" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5841d04b55e64deab20fa9ac3cd50389" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43235 } } }
I20260811 17:22:49.884256 2201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [sys.catalog]: This master's current role is: LEADER
I20260811 17:22:49.888684 2227 catalog_manager.cc:1259] Loaded cluster ID: fbab6d78e9684dad889bfbc7f6c34d20
I20260811 17:22:49.888761 2227 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389: loading cluster ID for follower catalog manager: success
I20260811 17:22:49.891345 2227 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389: acquiring CA information for follower catalog manager: success
I20260811 17:22:49.893038 2227 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 17:22:49.893198 2210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:22:49.895998 2210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:22:49.896597 2210 catalog_manager.cc:1259] Loaded cluster ID: fbab6d78e9684dad889bfbc7f6c34d20
I20260811 17:22:49.896673 2210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:22:49.897435 2210 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:22:49.897969 2210 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389: Loaded TSK: 0
I20260811 17:22:49.913233 2210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389: Generated new TSK 1
I20260811 17:22:49.913540 2210 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 17:22:49.968713 2194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:22:49.969039 2194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:22:49.969125 2194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:22:49.973654 2194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:22:49.973762 2194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:22:49.973862 2194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:22:49.979553 2194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:43025
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=40263
--tserver_master_addrs=127.9.157.190:43235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:22:49.980823 2194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:22:49.981954 2194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:22:49.994097 2245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:49.994138 2248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:49.994153 2244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:49.995072 2194 server_base.cc:1034] running on GCE node
I20260811 17:22:49.995702 2194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:22:49.996587 2194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:22:49.997814 2194 hybrid_clock.cc:648] HybridClock initialized: now 1786468969997768 us; error 69 us; skew 500 ppm
Aug 11 17:22:49 dist-test-slave-mjjr krb5kdc[1287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468969, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:22:50.001202 2194 init.cc:377] Logged in from keytab as kudu/127.9.157.129@KRBTEST.COM (short username kudu)
I20260811 17:22:50.002466 2194 webserver.cc:460] Webserver started at http://127.9.157.129:40263/ using document root <none> and password file <none>
I20260811 17:22:50.003139 2194 fs_manager.cc:359] Metadata directory not provided
I20260811 17:22:50.003242 2194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:22:50.008236 2194 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 17:22:50.011054 2258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:50.012923 2194 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:22:50.013113 2194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "b46d01b7d14947f9931fdea29f0f8e3c"
format_stamp: "Formatted at 2026-08-11 17:22:28 on dist-test-slave-mjjr"
server_key: "631d8b2cf58a98dfe5d8d5987af0c263"
server_key_iv: "776d93aa1f25f09dab2f5478917eadf3"
server_key_version: "encryptionkey@0"
I20260811 17:22:50.013978 2194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:22:50.036135 2194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:22:50.040802 2194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:22:50.041035 2194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:22:50.041884 2194 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:22:50.043542 2271 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 17:22:50.048774 2194 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 17:22:50.048966 2194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260811 17:22:50.049103 2194 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 17:22:50.051369 2194 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 17:22:50.051452 2194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 17:22:50.051771 2271 tablet_bootstrap.cc:492] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c: Bootstrap starting.
I20260811 17:22:50.064769 2194 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:43025
I20260811 17:22:50.064822 2422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:43025 every 8 connection(s)
I20260811 17:22:50.066273 2194 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:22:50.072950 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 2194
I20260811 17:22:50.073546 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:40213
--local_ip_for_outbound_sockets=127.9.157.130
--tserver_master_addrs=127.9.157.190:43235
--webserver_port=44369
--webserver_interface=127.9.157.130
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:22:50.077278 2271 log.cc:824] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c: Log is configured to *not* fsync() on all Append() calls
Aug 11 17:22:50 dist-test-slave-mjjr krb5kdc[1287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468969, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:22:50.085580 2426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:50.068623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.129:60823)
0811 17:22:50.069334 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:50.069391 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:50.070241 (+ 850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:50.071888 (+ 1647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:50.071895 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:50.072268 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:22:50.072755 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:50.072779 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.074344 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.074346 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:50.075218 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:50.075228 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.076266 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.076270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:50.076489 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:50.077289 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:22:50.077318 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:22:50.079452 (+ 2134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:22:50.081886 (+ 2434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:50.081892 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:50.081907 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:50.082185 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:50.082592 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:50.082595 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:50.082597 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:50.082747 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:50.083565 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:22:50.083572 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:22:50.083928 (+ 356us) client_negotiation.cc:769] Sending connection context
0811 17:22:50.084159 (+ 231us) client_negotiation.cc:240] Negotiation successful
0811 17:22:50.084587 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":153,"threads_started":1}
I20260811 17:22:50.085563 2425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:50.068706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:60823 (local address 127.9.157.190:43235)
0811 17:22:50.069211 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:50.069240 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:50.069537 (+ 297us) server_negotiation.cc:407] Connection header received
0811 17:22:50.070798 (+ 1261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:50.070819 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:50.071339 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:50.071795 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:50.072966 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.074158 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.075363 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.076112 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.076323 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:50.079784 (+ 3461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:50.079790 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:50.079806 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:50.079845 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:50.081721 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:50.082339 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:50.082345 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:50.082349 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:50.082482 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:50.082870 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:50.082878 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:50.082879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:50.083330 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:50.083583 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:50.084279 (+ 696us) server_negotiation.cc:299] Negotiation successful
0811 17:22:50.084587 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":158,"threads_started":1}
I20260811 17:22:50.088482 2423 heartbeater.cc:347] Connected to a master server at 127.9.157.190:43235
I20260811 17:22:50.088766 2423 heartbeater.cc:464] Registering TS with master...
I20260811 17:22:50.089316 2423 heartbeater.cc:511] Master 127.9.157.190:43235 requested a full tablet report, sending...
I20260811 17:22:50.089820 2271 tablet_bootstrap.cc:492] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 17:22:50.090395 2271 tablet_bootstrap.cc:492] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c: Bootstrap complete.
I20260811 17:22:50.091990 2147 ts_manager.cc:194] Registered new tserver with Master: b46d01b7d14947f9931fdea29f0f8e3c (127.9.157.129:43025)
I20260811 17:22:50.092259 2271 ts_tablet_manager.cc:1397] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c: Time spent bootstrapping tablet: real 0.041s user 0.028s sys 0.010s
I20260811 17:22:50.095798 2147 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.129@KRBTEST.COM'} at 127.9.157.129:60823
I20260811 17:22:50.095783 2271 raft_consensus.cc:348] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } } peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } }
I20260811 17:22:50.096413 2271 raft_consensus.cc:729] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b46d01b7d14947f9931fdea29f0f8e3c, State: Initialized, Role: FOLLOWER
I20260811 17:22:50.097024 2271 consensus_queue.cc:260] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } } peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } }
I20260811 17:22:50.098204 2271 ts_tablet_manager.cc:1428] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 17:22:50.098870 2423 heartbeater.cc:503] Master 127.9.157.190:43235 was elected leader, sending a full tablet report...
I20260811 17:22:50.102675 2430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:50.096026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37970 (local address 127.9.157.129:43025)
0811 17:22:50.097004 (+ 978us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:50.097017 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:50.097040 (+ 23us) server_negotiation.cc:407] Connection header received
0811 17:22:50.097108 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:50.097113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:50.097838 (+ 725us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:50.098006 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:22:50.098518 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.099510 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.101453 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.101972 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.102065 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:50.102175 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:50.102481 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 17:22:50.102563 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":870,"thread_start_us":118,"threads_started":1}
I20260811 17:22:50.102751 2429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:50.095079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:43025 (local address 127.0.0.1:37970)
0811 17:22:50.096336 (+ 1257us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:50.096537 (+ 201us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:50.096690 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:50.098165 (+ 1475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:50.098172 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:50.098197 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:22:50.098339 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:50.098347 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.099689 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.099694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:50.101317 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:50.101333 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.102116 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.102120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:50.102259 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:50.102307 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 17:22:50.102427 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 17:22:50.102565 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1034,"thread_start_us":135,"threads_started":1}
I20260811 17:22:50.104104 2328 tablet_service.cc:1514] Processing DeleteTablet for tablet 4775c95a1a524e0ab50c8775a233a16d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:22:29 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:37970
I20260811 17:22:50.105029 2434 tablet_replica.cc:330] stopping tablet replica
I20260811 17:22:50.105368 2434 raft_consensus.cc:2229] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:22:50.105640 2434 raft_consensus.cc:2258] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:22:50.106747 2434 ts_tablet_manager.cc:1905] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:22:50.110729 2434 ts_tablet_manager.cc:1918] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:22:50.110891 2434 log.cc:1196] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/wal/wals/4775c95a1a524e0ab50c8775a233a16d
I20260811 17:22:50.111184 2434 ts_tablet_manager.cc:1939] T 4775c95a1a524e0ab50c8775a233a16d P b46d01b7d14947f9931fdea29f0f8e3c: Deleting consensus metadata
I20260811 17:22:50.113103 2124 catalog_manager.cc:4841] TS b46d01b7d14947f9931fdea29f0f8e3c (127.9.157.129:43025): tablet 4775c95a1a524e0ab50c8775a233a16d (table test-table [id=c17ebc5f62dc4217bdac5c7f37701e2e]) successfully deleted
W20260811 17:22:50.217588 2427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:22:50.217971 2427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:22:50.218080 2427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:22:50.223167 2427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:22:50.223292 2427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:22:50.223402 2427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:22:50.229697 2427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:40213
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=44369
--tserver_master_addrs=127.9.157.190:43235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:22:50.231038 2427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:22:50.232272 2427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:22:50.243415 2452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:50.243484 2456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:50.243415 2450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:50.243759 2427 server_base.cc:1034] running on GCE node
I20260811 17:22:50.244359 2427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:22:50.245121 2427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:22:50.246332 2427 hybrid_clock.cc:648] HybridClock initialized: now 1786468970246317 us; error 28 us; skew 500 ppm
Aug 11 17:22:50 dist-test-slave-mjjr krb5kdc[1287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468970, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:22:50.249612 2427 init.cc:377] Logged in from keytab as kudu/127.9.157.130@KRBTEST.COM (short username kudu)
I20260811 17:22:50.250797 2427 webserver.cc:460] Webserver started at http://127.9.157.130:44369/ using document root <none> and password file <none>
I20260811 17:22:50.251472 2427 fs_manager.cc:359] Metadata directory not provided
I20260811 17:22:50.251619 2427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:22:50.256177 2427 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 17:22:50.258455 2468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:50.259917 2427 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:22:50.260032 2427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "87116910e3734e87b64cd77012c35417"
format_stamp: "Formatted at 2026-08-11 17:22:28 on dist-test-slave-mjjr"
server_key: "661f32eccb2e703c8e387ba31498acd3"
server_key_iv: "084cc39eb3ba48895a592a345887e9fc"
server_key_version: "encryptionkey@0"
I20260811 17:22:50.260619 2427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:22:50.276865 2427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:22:50.280340 2427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:22:50.280519 2427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:22:50.281141 2427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:22:50.282042 2427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:22:50.282100 2427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:50.282146 2427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:22:50.282161 2427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:50.292897 2427 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:40213
I20260811 17:22:50.292922 2656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:40213 every 8 connection(s)
I20260811 17:22:50.294066 2427 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:22:50.294570 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 2427
I20260811 17:22:50.295159 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:36241
--local_ip_for_outbound_sockets=127.9.157.131
--tserver_master_addrs=127.9.157.190:43235
--webserver_port=41123
--webserver_interface=127.9.157.131
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 17:22:50 dist-test-slave-mjjr krb5kdc[1287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468970, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:22:50.310590 2425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:50.296331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:50109 (local address 127.9.157.190:43235)
0811 17:22:50.296507 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:50.296511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:50.297497 (+ 986us) server_negotiation.cc:407] Connection header received
0811 17:22:50.298939 (+ 1442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:50.298945 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:50.299018 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:50.299131 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:50.300545 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.301481 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.302732 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.303188 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.303342 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:50.306064 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:50.306071 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:50.306076 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:50.306116 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:50.308131 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:50.308749 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:50.308755 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:50.308758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:50.308867 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:50.309287 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:50.309291 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:50.309292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:50.309544 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:50.309651 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:50.310244 (+ 593us) server_negotiation.cc:299] Negotiation successful
0811 17:22:50.310405 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 17:22:50.311532 2663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:50.296779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.130:50109)
0811 17:22:50.297305 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:50.297364 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:50.298702 (+ 1338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:50.299327 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:50.299335 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:50.299835 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:22:50.300357 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:50.300372 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.301646 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.301650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:50.302588 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:50.302598 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.303386 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.303389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:50.303545 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:50.304328 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:22:50.304346 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:22:50.305853 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:22:50.308299 (+ 2446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:50.308306 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:50.308322 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:50.308623 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:50.309004 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:50.309007 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:50.309008 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:50.309176 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:50.309683 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:22:50.309690 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:22:50.309986 (+ 296us) client_negotiation.cc:769] Sending connection context
0811 17:22:50.310178 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 17:22:50.310484 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":149,"threads_started":1}
I20260811 17:22:50.312949 2657 heartbeater.cc:347] Connected to a master server at 127.9.157.190:43235
I20260811 17:22:50.313215 2657 heartbeater.cc:464] Registering TS with master...
I20260811 17:22:50.313776 2657 heartbeater.cc:511] Master 127.9.157.190:43235 requested a full tablet report, sending...
I20260811 17:22:50.314956 2147 ts_manager.cc:194] Registered new tserver with Master: 87116910e3734e87b64cd77012c35417 (127.9.157.130:40213)
I20260811 17:22:50.315593 2147 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.130@KRBTEST.COM'} at 127.9.157.130:50109
W20260811 17:22:50.422773 2662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:22:50.423158 2662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:22:50.423249 2662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:22:50.427604 2662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:22:50.427759 2662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:22:50.427860 2662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:22:50.434634 2662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:36241
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=41123
--tserver_master_addrs=127.9.157.190:43235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:22:50.436017 2662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:22:50.437192 2662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:22:50.448654 2674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:22:50.448711 2673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:50.448777 2662 server_base.cc:1034] running on GCE node
W20260811 17:22:50.448711 2676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:22:50.449491 2662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:22:50.450222 2662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:22:50.451424 2662 hybrid_clock.cc:648] HybridClock initialized: now 1786468970451386 us; error 53 us; skew 500 ppm
Aug 11 17:22:50 dist-test-slave-mjjr krb5kdc[1287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468970, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:22:50.454825 2662 init.cc:377] Logged in from keytab as kudu/127.9.157.131@KRBTEST.COM (short username kudu)
I20260811 17:22:50.455976 2662 webserver.cc:460] Webserver started at http://127.9.157.131:41123/ using document root <none> and password file <none>
I20260811 17:22:50.456626 2662 fs_manager.cc:359] Metadata directory not provided
I20260811 17:22:50.456740 2662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:22:50.461174 2662 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 17:22:50.463552 2691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:50.465284 2662 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:22:50.465468 2662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "f5ca2cdc8d61405080d925fa399fc98b"
format_stamp: "Formatted at 2026-08-11 17:22:29 on dist-test-slave-mjjr"
server_key: "50746328f5c37fd65a3fb7a4898d0010"
server_key_iv: "68c9a7a7ec8b1cee68c5c4e33c363df2"
server_key_version: "encryptionkey@0"
I20260811 17:22:50.466274 2662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:22:50.484053 2662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:22:50.488543 2662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:22:50.488834 2662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:22:50.489588 2662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:22:50.490602 2662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:22:50.490690 2662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:50.490782 2662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:22:50.490845 2662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:22:50.502161 2662 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:36241
I20260811 17:22:50.502200 2859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:36241 every 8 connection(s)
I20260811 17:22:50.503378 2662 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:22:50.505128 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 2662
Aug 11 17:22:50 dist-test-slave-mjjr krb5kdc[1287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468970, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:22:50.515115 2866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:50.506034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46202 (local address 127.9.157.190:43235)
0811 17:22:50.506258 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:50.506261 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:50.506404 (+ 143us) server_negotiation.cc:407] Connection header received
0811 17:22:50.506609 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:50.506612 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:50.506661 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:50.506795 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:50.507901 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.508463 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.509388 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.509636 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.509752 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:50.511461 (+ 1709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:50.511465 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:50.511469 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:50.511504 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:50.513240 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:50.513713 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:50.513723 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:50.513725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:50.513833 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:50.514168 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:50.514174 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:50.514178 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:50.514412 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:50.514658 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:50.514826 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 17:22:50.514939 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":61,"threads_started":1}
I20260811 17:22:50.519337 2425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:50.505743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:52825 (local address 127.9.157.190:43235)
0811 17:22:50.505946 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:50.505953 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:50.507067 (+ 1114us) server_negotiation.cc:407] Connection header received
0811 17:22:50.508659 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:50.508662 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:50.508715 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:50.508796 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:50.510087 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.510713 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.512023 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.512254 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.512370 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:50.515305 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:50.515309 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:50.515311 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:50.515336 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:50.517051 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:50.517684 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:50.517687 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:50.517688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:50.517772 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:50.518141 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:50.518144 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:50.518145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:50.518367 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:50.518465 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:50.519051 (+ 586us) server_negotiation.cc:299] Negotiation successful
0811 17:22:50.519177 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260811 17:22:50.520371 2867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:50.506233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.131:52825)
0811 17:22:50.506905 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:50.506959 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:50.508428 (+ 1469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:50.509005 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:50.509013 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:50.509399 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:22:50.509929 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:50.509945 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.510861 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.510867 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:50.511893 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:50.511903 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.512438 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.512443 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:50.512620 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:50.513196 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:22:50.513215 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:22:50.515080 (+ 1865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:22:50.517195 (+ 2115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:50.517206 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:50.517224 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:50.517566 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:50.517866 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:22:50.517868 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:22:50.517869 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:22:50.518014 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:22:50.518475 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:22:50.518481 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:22:50.518744 (+ 263us) client_negotiation.cc:769] Sending connection context
0811 17:22:50.518972 (+ 228us) client_negotiation.cc:240] Negotiation successful
0811 17:22:50.519265 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":206,"threads_started":1}
I20260811 17:22:50.521668 2861 heartbeater.cc:347] Connected to a master server at 127.9.157.190:43235
I20260811 17:22:50.521929 2861 heartbeater.cc:464] Registering TS with master...
I20260811 17:22:50.522450 2861 heartbeater.cc:511] Master 127.9.157.190:43235 requested a full tablet report, sending...
I20260811 17:22:50.523581 2147 ts_manager.cc:194] Registered new tserver with Master: f5ca2cdc8d61405080d925fa399fc98b (127.9.157.131:36241)
I20260811 17:22:50.524354 2147 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.131@KRBTEST.COM'} at 127.9.157.131:52825
I20260811 17:22:50.530372 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:22:50.534680 2425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:50.531101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46204 (local address 127.9.157.190:43235)
0811 17:22:50.531242 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:50.531246 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:50.531496 (+ 250us) server_negotiation.cc:407] Connection header received
0811 17:22:50.531715 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:50.531718 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:50.531841 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:50.531940 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:22:50.532217 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.532737 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.533586 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.533796 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.533873 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:50.534173 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:22:50.534334 (+ 161us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 17:22:50.534559 (+ 225us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46204: token expired
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 17:22:50.534793 2425 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46204: token expired
W20260811 17:22:50.535164 2039 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.9.157.190:43235)
I20260811 17:22:50.542909 2425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:50.535579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46208 (local address 127.9.157.190:43235)
0811 17:22:50.535732 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:50.535738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:50.535753 (+ 15us) server_negotiation.cc:407] Connection header received
0811 17:22:50.535910 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:50.535914 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:50.535969 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:50.536058 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:22:50.536703 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.537228 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.538040 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.538257 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.538364 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:50.539811 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:22:50.539814 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:22:50.539816 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:22:50.539841 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:22:50.541161 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:50.541604 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:50.541608 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:50.541609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:50.541686 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:22:50.541976 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:22:50.541980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:22:50.541982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:22:50.542215 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:22:50.542359 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:50.542603 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 17:22:50.542731 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 17:22:50.546027 2147 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46208:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:22:50.549647 2147 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:22:50.558732 2328 tablet_service.cc:1467] Processing CreateTablet for tablet f74b4708125d46ee9b2676097b48f984 (DEFAULT_TABLE table=test-table [id=5ae45830beed4b8eba5b1f452574a8f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:22:50.559520 2328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f74b4708125d46ee9b2676097b48f984. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:22:50.562717 2271 tablet_bootstrap.cc:492] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c: Bootstrap starting.
I20260811 17:22:50.563064 2874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:50.557444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:36241 (local address 127.0.0.1:41618)
0811 17:22:50.557781 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:50.557795 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:50.557888 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:50.558735 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:50.558738 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:50.558762 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:22:50.558874 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:50.558884 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.560502 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.560508 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:50.561447 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:50.561459 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.562612 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.562616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:50.562717 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:50.562745 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 17:22:50.562826 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 17:22:50.562909 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":66,"thread_start_us":135,"threads_started":1}
I20260811 17:22:50.563670 2875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:50.557956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41618 (local address 127.9.157.131:36241)
0811 17:22:50.558346 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:50.558357 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:50.558376 (+ 19us) server_negotiation.cc:407] Connection header received
0811 17:22:50.558441 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:50.558445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:50.558582 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:50.558751 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:22:50.559070 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.560389 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.561592 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.562474 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.562584 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:50.562763 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:50.563358 (+ 595us) server_negotiation.cc:299] Negotiation successful
0811 17:22:50.563494 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":177,"threads_started":1}
I20260811 17:22:50.563810 2271 tablet_bootstrap.cc:654] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c: Neither blocks nor log segments found. Creating new log.
I20260811 17:22:50.564368 2429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:50.557151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:40213 (local address 127.0.0.1:48508)
0811 17:22:50.557348 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:50.557367 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:50.557451 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:50.558332 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:50.558335 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:50.558356 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:22:50.558466 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:50.558472 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.560152 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.560158 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:50.561447 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:50.561459 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.563899 (+ 2440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.563903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:50.564050 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:50.564079 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 17:22:50.564135 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 17:22:50.564230 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":112}
I20260811 17:22:50.565224 2873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:50.557347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48508 (local address 127.9.157.130:40213)
0811 17:22:50.557728 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:50.557743 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:50.557769 (+ 26us) server_negotiation.cc:407] Connection header received
0811 17:22:50.557863 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:50.557869 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:50.558129 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:50.558637 (+ 508us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:22:50.558862 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.560007 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.562838 (+ 2831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.563745 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.564678 (+ 933us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:50.564843 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:50.564902 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 17:22:50.565030 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":176,"threads_started":1}
I20260811 17:22:50.565522 2271 tablet_bootstrap.cc:492] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c: No bootstrap required, opened a new log
I20260811 17:22:50.565627 2271 ts_tablet_manager.cc:1397] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 17:22:50.566103 2271 raft_consensus.cc:348] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } } peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } }
I20260811 17:22:50.566267 2271 raft_consensus.cc:374] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:22:50.566321 2271 raft_consensus.cc:729] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b46d01b7d14947f9931fdea29f0f8e3c, State: Initialized, Role: FOLLOWER
I20260811 17:22:50.566481 2271 consensus_queue.cc:260] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } } peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } }
I20260811 17:22:50.566910 2271 ts_tablet_manager.cc:1428] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 17:22:50.567322 2760 tablet_service.cc:1467] Processing CreateTablet for tablet f74b4708125d46ee9b2676097b48f984 (DEFAULT_TABLE table=test-table [id=5ae45830beed4b8eba5b1f452574a8f9]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 17:22:50.568131 2424 tablet.cc:2369] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:22:50.568696 2760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f74b4708125d46ee9b2676097b48f984. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:22:50.568246 2542 tablet_service.cc:1467] Processing CreateTablet for tablet f74b4708125d46ee9b2676097b48f984 (DEFAULT_TABLE table=test-table [id=5ae45830beed4b8eba5b1f452574a8f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:22:50.569355 2542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f74b4708125d46ee9b2676097b48f984. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:22:50.578063 2879 tablet_bootstrap.cc:492] T f74b4708125d46ee9b2676097b48f984 P 87116910e3734e87b64cd77012c35417: Bootstrap starting.
I20260811 17:22:50.578940 2880 tablet_bootstrap.cc:492] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b: Bootstrap starting.
I20260811 17:22:50.581323 2880 tablet_bootstrap.cc:654] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b: Neither blocks nor log segments found. Creating new log.
I20260811 17:22:50.581513 2879 tablet_bootstrap.cc:654] T f74b4708125d46ee9b2676097b48f984 P 87116910e3734e87b64cd77012c35417: Neither blocks nor log segments found. Creating new log.
I20260811 17:22:50.582667 2880 log.cc:824] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b: Log is configured to *not* fsync() on all Append() calls
I20260811 17:22:50.582620 2879 log.cc:824] T f74b4708125d46ee9b2676097b48f984 P 87116910e3734e87b64cd77012c35417: Log is configured to *not* fsync() on all Append() calls
I20260811 17:22:50.585139 2879 tablet_bootstrap.cc:492] T f74b4708125d46ee9b2676097b48f984 P 87116910e3734e87b64cd77012c35417: No bootstrap required, opened a new log
I20260811 17:22:50.585150 2880 tablet_bootstrap.cc:492] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b: No bootstrap required, opened a new log
I20260811 17:22:50.585412 2880 ts_tablet_manager.cc:1397] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 17:22:50.585412 2879 ts_tablet_manager.cc:1397] T f74b4708125d46ee9b2676097b48f984 P 87116910e3734e87b64cd77012c35417: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 17:22:50.588927 2880 raft_consensus.cc:348] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } } peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } }
I20260811 17:22:50.588927 2879 raft_consensus.cc:348] T f74b4708125d46ee9b2676097b48f984 P 87116910e3734e87b64cd77012c35417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } } peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } }
I20260811 17:22:50.589218 2880 raft_consensus.cc:374] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:22:50.589244 2879 raft_consensus.cc:374] T f74b4708125d46ee9b2676097b48f984 P 87116910e3734e87b64cd77012c35417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:22:50.589258 2880 raft_consensus.cc:729] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ca2cdc8d61405080d925fa399fc98b, State: Initialized, Role: FOLLOWER
I20260811 17:22:50.589314 2879 raft_consensus.cc:729] T f74b4708125d46ee9b2676097b48f984 P 87116910e3734e87b64cd77012c35417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87116910e3734e87b64cd77012c35417, State: Initialized, Role: FOLLOWER
I20260811 17:22:50.589821 2879 consensus_queue.cc:260] T f74b4708125d46ee9b2676097b48f984 P 87116910e3734e87b64cd77012c35417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } } peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } }
I20260811 17:22:50.589826 2880 consensus_queue.cc:260] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } } peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } }
I20260811 17:22:50.590616 2657 heartbeater.cc:503] Master 127.9.157.190:43235 was elected leader, sending a full tablet report...
I20260811 17:22:50.590617 2861 heartbeater.cc:503] Master 127.9.157.190:43235 was elected leader, sending a full tablet report...
I20260811 17:22:50.591041 2879 ts_tablet_manager.cc:1428] T f74b4708125d46ee9b2676097b48f984 P 87116910e3734e87b64cd77012c35417: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 17:22:50.591725 2880 ts_tablet_manager.cc:1428] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260811 17:22:50.754667 2862 tablet.cc:2369] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:22:50.761487 2887 raft_consensus.cc:482] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:22:50.761755 2887 raft_consensus.cc:504] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } } peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } }
I20260811 17:22:50.763121 2887 leader_election.cc:288] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87116910e3734e87b64cd77012c35417 (127.9.157.130:40213), b46d01b7d14947f9931fdea29f0f8e3c (127.9.157.129:43025)
I20260811 17:22:50.767551 2867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:50.763358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:40213 (local address 127.9.157.131:59881)
0811 17:22:50.763518 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:50.763537 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:50.763675 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:50.764048 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:50.764051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:50.764071 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:22:50.764229 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:50.764244 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.765256 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.765268 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:50.766283 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:50.766295 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.767033 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.767037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:50.767164 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:50.767204 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 17:22:50.767290 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 17:22:50.767379 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 17:22:50.767560 2873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:50.763600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:59881 (local address 127.9.157.130:40213)
0811 17:22:50.763777 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:50.763783 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:50.763799 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:22:50.763859 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:50.763862 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:50.763917 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:50.763999 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:22:50.764410 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.765124 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.766434 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.766883 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.766988 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:50.767032 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:50.767324 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 17:22:50.767421 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 17:22:50.767787 2890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:50.763612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:54049 (local address 127.9.157.129:43025)
0811 17:22:50.763922 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:50.763926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:50.764021 (+ 95us) server_negotiation.cc:407] Connection header received
0811 17:22:50.764214 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:50.764217 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:50.764273 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:50.764364 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:22:50.764689 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.765488 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.766577 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.767117 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.767245 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:50.767310 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:50.767513 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 17:22:50.767611 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":64,"threads_started":1}
I20260811 17:22:50.767805 2889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:50.763474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:43025 (local address 127.9.157.131:54049)
0811 17:22:50.763923 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:50.763936 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:22:50.764063 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:22:50.764394 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:22:50.764397 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:22:50.764415 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:22:50.764558 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:50.764566 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.765610 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.765612 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:50.766417 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:22:50.766429 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.767241 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.767243 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:22:50.767372 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:50.767396 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 17:22:50.767473 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 17:22:50.767539 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":145,"threads_started":1}
I20260811 17:22:50.768540 2357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f74b4708125d46ee9b2676097b48f984" candidate_uuid: "f5ca2cdc8d61405080d925fa399fc98b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" is_pre_election: true
I20260811 17:22:50.768827 2578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f74b4708125d46ee9b2676097b48f984" candidate_uuid: "f5ca2cdc8d61405080d925fa399fc98b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87116910e3734e87b64cd77012c35417" is_pre_election: true
I20260811 17:22:50.768955 2357 raft_consensus.cc:2454] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ca2cdc8d61405080d925fa399fc98b in term 0.
I20260811 17:22:50.769129 2578 raft_consensus.cc:2454] T f74b4708125d46ee9b2676097b48f984 P 87116910e3734e87b64cd77012c35417 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ca2cdc8d61405080d925fa399fc98b in term 0.
I20260811 17:22:50.769511 2696 leader_election.cc:302] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b46d01b7d14947f9931fdea29f0f8e3c, f5ca2cdc8d61405080d925fa399fc98b; no voters:
I20260811 17:22:50.769876 2887 raft_consensus.cc:2781] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:22:50.769982 2887 raft_consensus.cc:482] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:22:50.770037 2887 raft_consensus.cc:3037] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:22:50.771262 2887 raft_consensus.cc:504] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } } peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } }
I20260811 17:22:50.771719 2887 leader_election.cc:288] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b [CANDIDATE]: Term 1 election: Requested vote from peers 87116910e3734e87b64cd77012c35417 (127.9.157.130:40213), b46d01b7d14947f9931fdea29f0f8e3c (127.9.157.129:43025)
I20260811 17:22:50.772112 2578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f74b4708125d46ee9b2676097b48f984" candidate_uuid: "f5ca2cdc8d61405080d925fa399fc98b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87116910e3734e87b64cd77012c35417"
I20260811 17:22:50.772169 2357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f74b4708125d46ee9b2676097b48f984" candidate_uuid: "f5ca2cdc8d61405080d925fa399fc98b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b46d01b7d14947f9931fdea29f0f8e3c"
I20260811 17:22:50.772275 2578 raft_consensus.cc:3037] T f74b4708125d46ee9b2676097b48f984 P 87116910e3734e87b64cd77012c35417 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:22:50.772364 2357 raft_consensus.cc:3037] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:22:50.773555 2357 raft_consensus.cc:2454] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ca2cdc8d61405080d925fa399fc98b in term 1.
I20260811 17:22:50.773562 2578 raft_consensus.cc:2454] T f74b4708125d46ee9b2676097b48f984 P 87116910e3734e87b64cd77012c35417 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ca2cdc8d61405080d925fa399fc98b in term 1.
I20260811 17:22:50.773939 2696 leader_election.cc:302] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b46d01b7d14947f9931fdea29f0f8e3c, f5ca2cdc8d61405080d925fa399fc98b; no voters:
I20260811 17:22:50.774216 2887 raft_consensus.cc:2781] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:22:50.774513 2887 raft_consensus.cc:686] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b [term 1 LEADER]: Becoming Leader. State: Replica: f5ca2cdc8d61405080d925fa399fc98b, State: Running, Role: LEADER
I20260811 17:22:50.774901 2887 consensus_queue.cc:237] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } } peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } }
I20260811 17:22:50.778611 2147 catalog_manager.cc:5495] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b reported cstate change: term changed from 0 to 1, leader changed from <none> to f5ca2cdc8d61405080d925fa399fc98b (127.9.157.131). New cstate: current_term: 1 leader_uuid: "f5ca2cdc8d61405080d925fa399fc98b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } health_report { overall_health: UNKNOWN } } }
W20260811 17:22:50.795572 2661 tablet.cc:2369] T f74b4708125d46ee9b2676097b48f984 P 87116910e3734e87b64cd77012c35417: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:22:50.815968 2875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:50.812080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41628 (local address 127.9.157.131:36241)
0811 17:22:50.812259 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 17:22:50.812266 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:22:50.812285 (+ 19us) server_negotiation.cc:407] Connection header received
0811 17:22:50.812377 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:22:50.812380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:22:50.812437 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:22:50.812531 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:22:50.812771 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.813280 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.814182 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:22:50.814405 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:22:50.814497 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:22:50.814796 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:22:50.815275 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:22:50.815372 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 17:22:50.815600 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 17:22:50.815771 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 17:22:50.823807 2578 raft_consensus.cc:1261] T f74b4708125d46ee9b2676097b48f984 P 87116910e3734e87b64cd77012c35417 [term 1 FOLLOWER]: Refusing update from remote peer f5ca2cdc8d61405080d925fa399fc98b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:22:50.823804 2357 raft_consensus.cc:1261] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [term 1 FOLLOWER]: Refusing update from remote peer f5ca2cdc8d61405080d925fa399fc98b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:22:50.824874 2887 consensus_queue.cc:1035] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:22:50.825214 2893 consensus_queue.cc:1035] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b [LEADER]: Connected to new peer: Peer: permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:22:50.834668 2900 mvcc.cc:205] Tried to move back new op lower bound from 7317376904484859904 to 7317376904296751104. Current Snapshot: MvccSnapshot[applied={T|T < 7317376904484859904}]
I20260811 17:22:50.849031 2144 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46208:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:22:50.849291 2144 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46208:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:22:50.852272 2144 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389: Sending DeleteTablet for 3 replicas of tablet f74b4708125d46ee9b2676097b48f984
I20260811 17:22:50.853255 2542 tablet_service.cc:1514] Processing DeleteTablet for tablet f74b4708125d46ee9b2676097b48f984 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:22:50 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:48508
I20260811 17:22:50.853255 2760 tablet_service.cc:1514] Processing DeleteTablet for tablet f74b4708125d46ee9b2676097b48f984 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:22:50 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:41618
I20260811 17:22:50.853484 2328 tablet_service.cc:1514] Processing DeleteTablet for tablet f74b4708125d46ee9b2676097b48f984 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:22:50 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:37970
I20260811 17:22:50.853953 2917 tablet_replica.cc:330] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c: stopping tablet replica
I20260811 17:22:50.854128 2917 raft_consensus.cc:2229] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:22:50.854218 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 2194
I20260811 17:22:50.869647 2915 tablet_replica.cc:330] T f74b4708125d46ee9b2676097b48f984 P 87116910e3734e87b64cd77012c35417: stopping tablet replica
I20260811 17:22:50.870348 2916 tablet_replica.cc:330] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b: stopping tablet replica
I20260811 17:22:50.870690 2916 raft_consensus.cc:2229] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:22:50.871297 2916 raft_consensus.cc:2258] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:22:50.871665 2915 raft_consensus.cc:2229] T f74b4708125d46ee9b2676097b48f984 P 87116910e3734e87b64cd77012c35417 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:22:50.872064 2915 raft_consensus.cc:2258] T f74b4708125d46ee9b2676097b48f984 P 87116910e3734e87b64cd77012c35417 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:22:50.873396 2916 ts_tablet_manager.cc:1905] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:22:50.873740 2915 ts_tablet_manager.cc:1905] T f74b4708125d46ee9b2676097b48f984 P 87116910e3734e87b64cd77012c35417: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:22:50.876834 2915 ts_tablet_manager.cc:1918] T f74b4708125d46ee9b2676097b48f984 P 87116910e3734e87b64cd77012c35417: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:22:50.876972 2915 log.cc:1196] T f74b4708125d46ee9b2676097b48f984 P 87116910e3734e87b64cd77012c35417: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/wal/wals/f74b4708125d46ee9b2676097b48f984
I20260811 17:22:50.877338 2915 ts_tablet_manager.cc:1939] T f74b4708125d46ee9b2676097b48f984 P 87116910e3734e87b64cd77012c35417: Deleting consensus metadata
I20260811 17:22:50.877647 2916 ts_tablet_manager.cc:1918] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:22:50.877837 2916 log.cc:1196] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/wal/wals/f74b4708125d46ee9b2676097b48f984
W20260811 17:22:50.878093 2124 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.157.129:43025 (error 108)
I20260811 17:22:50.878361 2916 ts_tablet_manager.cc:1939] T f74b4708125d46ee9b2676097b48f984 P f5ca2cdc8d61405080d925fa399fc98b: Deleting consensus metadata
I20260811 17:22:50.878865 2121 catalog_manager.cc:4841] TS 87116910e3734e87b64cd77012c35417 (127.9.157.130:40213): tablet f74b4708125d46ee9b2676097b48f984 (table test-table [id=5ae45830beed4b8eba5b1f452574a8f9]) successfully deleted
I20260811 17:22:50.879469 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 2427
I20260811 17:22:50.880064 2429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:22:50.879709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:43025 (local address 127.0.0.1:37976)
0811 17:22:50.879888 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 17:22:50.879971 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.129:43025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260811 17:22:50.880230 2124 catalog_manager.cc:4584] TS b46d01b7d14947f9931fdea29f0f8e3c (127.9.157.129:43025): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f74b4708125d46ee9b2676097b48f984: Network error: Client connection negotiation failed: client connection to 127.9.157.129:43025: connect: Connection refused (error 111)
I20260811 17:22:50.882719 2123 catalog_manager.cc:4841] TS f5ca2cdc8d61405080d925fa399fc98b (127.9.157.131:36241): tablet f74b4708125d46ee9b2676097b48f984 (table test-table [id=5ae45830beed4b8eba5b1f452574a8f9]) successfully deleted
I20260811 17:22:50.894023 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 2662
I20260811 17:22:50.909935 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 2097
I20260811 17:23:10.930305 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:43235
--webserver_interface=127.9.157.190
--webserver_port=42677
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:43235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:23:11.055950 2920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:11.056257 2920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:11.056346 2920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:11.060556 2920 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 17:23:11.060619 2920 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 17:23:11.060635 2920 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 17:23:11.060649 2920 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:23:11.060668 2920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:11.060710 2920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:11.060745 2920 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:23:11.060765 2920 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:23:11.066377 2920 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:43235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:43235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=42677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:11.067618 2920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:11.068801 2920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:11.078022 2927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:11.078157 2920 server_base.cc:1034] running on GCE node
W20260811 17:23:11.078063 2932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:11.078029 2928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:11.079016 2920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:11.080403 2920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:11.081640 2920 hybrid_clock.cc:648] HybridClock initialized: now 1786468991081601 us; error 51 us; skew 500 ppm
Aug 11 17:23:11 dist-test-slave-mjjr krb5kdc[1287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468991, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:23:11.084900 2920 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:23:11.085968 2920 webserver.cc:460] Webserver started at http://127.9.157.190:42677/ using document root <none> and password file <none>
I20260811 17:23:11.086639 2920 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:11.086751 2920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:11.091266 2920 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:23:11.093890 2945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:11.095294 2920 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:23:11.095470 2920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "5841d04b55e64deab20fa9ac3cd50389"
format_stamp: "Formatted at 2026-08-11 17:22:28 on dist-test-slave-mjjr"
server_key: "c9477597b809516e48ece22892c49a0d"
server_key_iv: "8b139dffa22d611a384b01763ddfd88d"
server_key_version: "encryptionkey@0"
I20260811 17:23:11.096177 2920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:11.111833 2920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:11.120296 2920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:11.120564 2920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:11.128664 2920 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:43235
I20260811 17:23:11.129014 3024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:43235 every 8 connection(s)
I20260811 17:23:11.129921 2920 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:23:11.130103 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 2920
I20260811 17:23:11.130671 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:43025
--local_ip_for_outbound_sockets=127.9.157.129
--tserver_master_addrs=127.9.157.190:43235
--webserver_port=40263
--webserver_interface=127.9.157.129
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:23:11.134918 3028 sys_catalog.cc:263] Verifying existing consensus state
I20260811 17:23:11.136972 3028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389: Bootstrap starting.
I20260811 17:23:11.156124 3028 log.cc:824] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:11.168483 3028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 17:23:11.168954 3028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389: Bootstrap complete.
I20260811 17:23:11.173466 3028 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5841d04b55e64deab20fa9ac3cd50389" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43235 } }
I20260811 17:23:11.173949 3028 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5841d04b55e64deab20fa9ac3cd50389, State: Initialized, Role: FOLLOWER
I20260811 17:23:11.174548 3028 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5841d04b55e64deab20fa9ac3cd50389" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43235 } }
I20260811 17:23:11.174711 3028 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:23:11.174808 3028 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:23:11.174942 3028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [term 2 FOLLOWER]: Advancing to term 3
I20260811 17:23:11.177184 3028 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5841d04b55e64deab20fa9ac3cd50389" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43235 } }
I20260811 17:23:11.177573 3028 leader_election.cc:302] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5841d04b55e64deab20fa9ac3cd50389; no voters:
I20260811 17:23:11.177997 3028 leader_election.cc:288] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 17:23:11.178094 3034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 17:23:11.178376 3034 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [term 3 LEADER]: Becoming Leader. State: Replica: 5841d04b55e64deab20fa9ac3cd50389, State: Running, Role: LEADER
I20260811 17:23:11.178771 3034 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5841d04b55e64deab20fa9ac3cd50389" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43235 } }
I20260811 17:23:11.179477 3028 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:23:11.180451 3036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5841d04b55e64deab20fa9ac3cd50389. Latest consensus state: current_term: 3 leader_uuid: "5841d04b55e64deab20fa9ac3cd50389" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5841d04b55e64deab20fa9ac3cd50389" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43235 } } }
I20260811 17:23:11.180590 3036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:11.180512 3035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "5841d04b55e64deab20fa9ac3cd50389" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5841d04b55e64deab20fa9ac3cd50389" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43235 } } }
I20260811 17:23:11.180642 3035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389 [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:11.185320 3061 catalog_manager.cc:1259] Loaded cluster ID: fbab6d78e9684dad889bfbc7f6c34d20
I20260811 17:23:11.185421 3061 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389: loading cluster ID for follower catalog manager: success
I20260811 17:23:11.188257 3061 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5841d04b55e64deab20fa9ac3cd50389: acquiring CA information for follower catalog manager: success
F20260811 17:23:11.189693 3061 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe42bf719ad google::LogMessage::Fail() at ??:0
@ 0x7fe42bf7488a google::LogMessage::SendToLog() at ??:0
@ 0x7fe42bf714de google::LogMessage::Flush() at ??:0
@ 0x7fe42bf72a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe42ec3518d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fe4368d6989 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7fe4368b3023 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7fe4368b354b kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7fe4368ac652 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7fe4368ab9e9 _ZZN4kudu6master21CatalogManagerBgTasks4InitEvENKUlvE_clEv at ??:0
@ 0x7fe4368f2541 _ZNSt17_Function_handlerIFvvEZN4kudu6master21CatalogManagerBgTasks4InitEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fe42e1b4512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe42f7f86db start_thread at ??:0
@ 0x7fe42aa6f61f clone at ??:0
W20260811 17:23:11.281764 3031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:11.282056 3031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:11.282104 3031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:11.287004 3031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:11.287092 3031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:11.287159 3031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:23:11.293517 3031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:43025
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=40263
--tserver_master_addrs=127.9.157.190:43235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:11.294838 3031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:11.296212 3031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:11.308509 3089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:11.308512 3092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:11.308511 3090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:11.308984 3031 server_base.cc:1034] running on GCE node
I20260811 17:23:11.309549 3031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:11.310320 3031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:11.311519 3031 hybrid_clock.cc:648] HybridClock initialized: now 1786468991311500 us; error 42 us; skew 500 ppm
Aug 11 17:23:11 dist-test-slave-mjjr krb5kdc[1287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468991, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:23:11.314548 3031 init.cc:377] Logged in from keytab as kudu/127.9.157.129@KRBTEST.COM (short username kudu)
I20260811 17:23:11.315611 3031 webserver.cc:460] Webserver started at http://127.9.157.129:40263/ using document root <none> and password file <none>
I20260811 17:23:11.316316 3031 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:11.316387 3031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:11.320778 3031 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 17:23:11.323233 3103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:11.324733 3031 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:23:11.324887 3031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "b46d01b7d14947f9931fdea29f0f8e3c"
format_stamp: "Formatted at 2026-08-11 17:22:28 on dist-test-slave-mjjr"
server_key: "631d8b2cf58a98dfe5d8d5987af0c263"
server_key_iv: "776d93aa1f25f09dab2f5478917eadf3"
server_key_version: "encryptionkey@0"
I20260811 17:23:11.325510 3031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:11.361361 3031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:11.365140 3031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:11.365370 3031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:11.366144 3031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:23:11.367401 3114 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 17:23:11.371930 3031 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 17:23:11.372005 3031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 17:23:11.372062 3031 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 17:23:11.373924 3031 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 17:23:11.373992 3031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 17:23:11.374298 3114 tablet_bootstrap.cc:492] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c: Bootstrap starting.
I20260811 17:23:11.385516 3031 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:43025
I20260811 17:23:11.385859 3284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:43025 every 8 connection(s)
I20260811 17:23:11.387008 3031 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:23:11.390564 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 3031
I20260811 17:23:11.391086 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:40213
--local_ip_for_outbound_sockets=127.9.157.130
--tserver_master_addrs=127.9.157.190:43235
--webserver_port=44369
--webserver_interface=127.9.157.130
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:23:11.400581 3114 log.cc:824] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c: Log is configured to *not* fsync() on all Append() calls
W20260811 17:23:11.410959 3288 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:23:11.389648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.129:48541)
0811 17:23:11.390932 (+ 1284us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:11.390998 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:11.392583 (+ 1585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:11.409836 (+ 17253us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":156,"threads_started":1}
I20260811 17:23:11.411679 3114 tablet_bootstrap.cc:492] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260811 17:23:11.411782 3286 heartbeater.cc:644] Failed to heartbeat to 127.9.157.190:43235 (0 consecutive failures): Network error: Failed to ping master at 127.9.157.190:43235: Client connection negotiation failed: client connection to 127.9.157.190:43235: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 17:23:11.412180 3114 tablet_bootstrap.cc:492] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c: Bootstrap complete.
I20260811 17:23:11.412591 3288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:11.412304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.129:49867)
0811 17:23:11.412436 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:11.412515 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 17:23:11.413422 3288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:11.413140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.129:39073)
0811 17:23:11.413310 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:11.413361 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260811 17:23:11.413545 3114 ts_tablet_manager.cc:1397] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c: Time spent bootstrapping tablet: real 0.040s user 0.028s sys 0.007s
W20260811 17:23:11.413731 3286 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 17:23:11.416191 3114 raft_consensus.cc:348] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } } peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } }
I20260811 17:23:11.416631 3114 raft_consensus.cc:729] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b46d01b7d14947f9931fdea29f0f8e3c, State: Initialized, Role: FOLLOWER
I20260811 17:23:11.417171 3114 consensus_queue.cc:260] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } } peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } }
I20260811 17:23:11.418265 3114 ts_tablet_manager.cc:1428] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 17:23:11.418506 3288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:11.418167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.129:50651)
0811 17:23:11.418359 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:11.418420 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260811 17:23:11.419255 3288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:11.419006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.129:41123)
0811 17:23:11.419131 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:11.419186 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 17:23:11.520196 3289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:11.520498 3289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:11.520589 3289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:11.524912 3289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:11.524992 3289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:11.525055 3289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:23:11.530666 3289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:40213
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=44369
--tserver_master_addrs=127.9.157.190:43235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:11.531977 3289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:11.533069 3289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:11.543198 3302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:11.543262 3299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:11.543224 3300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:11.543457 3289 server_base.cc:1034] running on GCE node
I20260811 17:23:11.544094 3289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:11.544833 3289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:11.546053 3289 hybrid_clock.cc:648] HybridClock initialized: now 1786468991546031 us; error 32 us; skew 500 ppm
Aug 11 17:23:11 dist-test-slave-mjjr krb5kdc[1287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468991, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:23:11.549777 3289 init.cc:377] Logged in from keytab as kudu/127.9.157.130@KRBTEST.COM (short username kudu)
I20260811 17:23:11.550851 3289 webserver.cc:460] Webserver started at http://127.9.157.130:44369/ using document root <none> and password file <none>
I20260811 17:23:11.551533 3289 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:11.551646 3289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:11.555975 3289 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260811 17:23:11.558279 3319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:11.559618 3289 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:23:11.559859 3289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "87116910e3734e87b64cd77012c35417"
format_stamp: "Formatted at 2026-08-11 17:22:28 on dist-test-slave-mjjr"
server_key: "661f32eccb2e703c8e387ba31498acd3"
server_key_iv: "084cc39eb3ba48895a592a345887e9fc"
server_key_version: "encryptionkey@0"
I20260811 17:23:11.560472 3289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:11.575423 3289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:11.579170 3289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:11.579412 3289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:11.580132 3289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:23:11.581070 3289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:23:11.581146 3289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:11.581220 3289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:23:11.581269 3289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:11.591111 3289 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:40213
I20260811 17:23:11.591137 3488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:40213 every 8 connection(s)
I20260811 17:23:11.592278 3289 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:23:11.592769 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 3289
I20260811 17:23:11.593327 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:36241
--local_ip_for_outbound_sockets=127.9.157.131
--tserver_master_addrs=127.9.157.190:43235
--webserver_port=41123
--webserver_interface=127.9.157.131
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:23:11.596776 3491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:11.594680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.130:53265)
0811 17:23:11.595305 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:11.595445 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":127,"threads_started":1}
W20260811 17:23:11.597381 3489 heartbeater.cc:644] Failed to heartbeat to 127.9.157.190:43235 (0 consecutive failures): Network error: Failed to ping master at 127.9.157.190:43235: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
I20260811 17:23:11.598100 3491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:11.597835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.130:53525)
0811 17:23:11.597976 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:11.598039 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 17:23:11.598919 3491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:11.598703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.130:49829)
0811 17:23:11.598815 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:11.598860 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 17:23:11.599179 3489 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 17:23:11.715480 3492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:11.715821 3492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:11.715907 3492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:11.720131 3492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:11.720219 3492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:11.720283 3492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:23:11.725981 3492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:36241
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=41123
--tserver_master_addrs=127.9.157.190:43235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:11.727212 3492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:11.728341 3492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:11.738663 3503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:11.738663 3504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:11.738754 3492 server_base.cc:1034] running on GCE node
W20260811 17:23:11.738664 3507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:11.739946 3492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:11.740666 3492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:11.741876 3492 hybrid_clock.cc:648] HybridClock initialized: now 1786468991741855 us; error 32 us; skew 500 ppm
Aug 11 17:23:11 dist-test-slave-mjjr krb5kdc[1287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468991, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:23:11.745065 3492 init.cc:377] Logged in from keytab as kudu/127.9.157.131@KRBTEST.COM (short username kudu)
I20260811 17:23:11.746150 3492 webserver.cc:460] Webserver started at http://127.9.157.131:41123/ using document root <none> and password file <none>
I20260811 17:23:11.746834 3492 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:11.746915 3492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:11.751288 3492 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:23:11.753660 3518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:11.755055 3492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:23:11.755220 3492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "f5ca2cdc8d61405080d925fa399fc98b"
format_stamp: "Formatted at 2026-08-11 17:22:29 on dist-test-slave-mjjr"
server_key: "50746328f5c37fd65a3fb7a4898d0010"
server_key_iv: "68c9a7a7ec8b1cee68c5c4e33c363df2"
server_key_version: "encryptionkey@0"
I20260811 17:23:11.755887 3492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:11.783195 3492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:11.786933 3492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:11.787168 3492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:11.787889 3492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:23:11.788811 3492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:23:11.788869 3492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:11.788913 3492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:23:11.788959 3492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:11.798784 3492 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:36241
I20260811 17:23:11.798813 3694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:36241 every 8 connection(s)
I20260811 17:23:11.800029 3492 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:23:11.802747 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 3492
I20260811 17:23:11.804764 3699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:11.802379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.131:37491)
0811 17:23:11.802999 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:11.803171 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":151,"threads_started":1}
W20260811 17:23:11.805366 3696 heartbeater.cc:644] Failed to heartbeat to 127.9.157.190:43235 (0 consecutive failures): Network error: Failed to ping master at 127.9.157.190:43235: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
I20260811 17:23:11.806008 3699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:11.805761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.131:49501)
0811 17:23:11.805891 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:11.805943 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 17:23:11.806793 3699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:11.806563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.131:34605)
0811 17:23:11.806692 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:11.806741 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 17:23:11.807083 3696 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 17:23:12.420655 3703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:12.420174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.129:48673)
0811 17:23:12.420466 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:12.420547 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":97,"threads_started":1}
I20260811 17:23:12.600435 3704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:12.599907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.130:36447)
0811 17:23:12.600230 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:12.600303 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":83,"threads_started":1}
I20260811 17:23:12.809098 3705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:12.808307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.131:56931)
0811 17:23:12.808787 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:12.808923 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":158,"threads_started":1}
I20260811 17:23:13.206414 3707 raft_consensus.cc:482] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:23:13.206843 3707 raft_consensus.cc:504] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } } peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } }
I20260811 17:23:13.208345 3707 leader_election.cc:288] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 87116910e3734e87b64cd77012c35417 (127.9.157.130:40213), f5ca2cdc8d61405080d925fa399fc98b (127.9.157.131:36241)
Aug 11 17:23:13 dist-test-slave-mjjr krb5kdc[1287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468991, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for kudu/127.9.157.130@KRBTEST.COM
Aug 11 17:23:13 dist-test-slave-mjjr krb5kdc[1287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786468991, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for kudu/127.9.157.131@KRBTEST.COM
I20260811 17:23:13.224346 3715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:13.209058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:39853 (local address 127.9.157.130:40213)
0811 17:23:13.209436 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:13.209451 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:13.209491 (+ 40us) server_negotiation.cc:407] Connection header received
0811 17:23:13.210201 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:13.210209 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:13.210593 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:13.210951 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:13.212720 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:13.213729 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:13.214982 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:13.215481 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:13.215697 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:13.218715 (+ 3018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:13.218722 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:13.218737 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:23:13.218775 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:13.221204 (+ 2429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:13.221848 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:13.221853 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:13.221858 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:13.222000 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:13.222393 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:13.222397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:13.222399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:13.223024 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:13.223186 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:13.223848 (+ 662us) server_negotiation.cc:299] Negotiation successful
0811 17:23:13.224149 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":112,"threads_started":1}
I20260811 17:23:13.224328 3708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:13.208773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:40213 (local address 127.9.157.129:39853)
0811 17:23:13.209133 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:13.209150 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:13.209317 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:13.211034 (+ 1717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:13.211041 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:13.211573 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:23:13.212500 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:13.212523 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:13.213888 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:13.213892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:13.214845 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:13.214854 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:13.215634 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:13.215665 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:13.215883 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:13.216537 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:23:13.216555 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:23:13.218430 (+ 1875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:23:13.221363 (+ 2933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:23:13.221368 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:23:13.221371 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:23:13.221696 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:23:13.222117 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:23:13.222120 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:23:13.222122 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:23:13.222283 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:23:13.223155 (+ 872us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:23:13.223162 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:23:13.223457 (+ 295us) client_negotiation.cc:769] Sending connection context
0811 17:23:13.223722 (+ 265us) client_negotiation.cc:240] Negotiation successful
0811 17:23:13.224114 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":240,"spinlock_wait_cycles":24960,"thread_start_us":95,"threads_started":1}
I20260811 17:23:13.224468 3710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:13.208693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:36241 (local address 127.9.157.129:56899)
0811 17:23:13.209108 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:13.209126 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:13.209268 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:13.211034 (+ 1766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:13.211041 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:13.211825 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:23:13.212506 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:13.212523 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:13.213964 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:13.213969 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:13.214837 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:13.214848 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:13.215634 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:13.215662 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:13.215890 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:13.216664 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:23:13.216676 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:23:13.218577 (+ 1901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:23:13.221254 (+ 2677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:23:13.221261 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:23:13.221278 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:23:13.221673 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:23:13.222357 (+ 684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:23:13.222360 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:23:13.222362 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:23:13.222461 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:23:13.223562 (+ 1101us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:23:13.223565 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:23:13.223694 (+ 129us) client_negotiation.cc:769] Sending connection context
0811 17:23:13.223828 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 17:23:13.224223 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":332,"thread_start_us":97,"threads_started":1}
I20260811 17:23:13.224519 3711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:13.208871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:56899 (local address 127.9.157.131:36241)
0811 17:23:13.209524 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:13.209544 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:13.209579 (+ 35us) server_negotiation.cc:407] Connection header received
0811 17:23:13.210215 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:13.210220 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:13.210593 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:13.210950 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:13.212801 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:13.213821 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:13.214989 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:13.215482 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:13.215695 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:13.218857 (+ 3162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:13.218863 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:13.218876 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:23:13.218909 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:13.221087 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:13.221830 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:13.221838 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:13.221842 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:13.222208 (+ 366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:13.222591 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:13.222594 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:13.222596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:13.223106 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:13.223602 (+ 496us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:13.224005 (+ 403us) server_negotiation.cc:299] Negotiation successful
0811 17:23:13.224333 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":111,"threads_started":1}
W20260811 17:23:13.227692 3110 leader_election.cc:341] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f5ca2cdc8d61405080d925fa399fc98b (127.9.157.131:36241): Not found: Tablet not found: f74b4708125d46ee9b2676097b48f984
W20260811 17:23:13.228003 3104 leader_election.cc:341] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 87116910e3734e87b64cd77012c35417 (127.9.157.130:40213): Not found: Tablet not found: f74b4708125d46ee9b2676097b48f984
I20260811 17:23:13.228070 3104 leader_election.cc:302] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b46d01b7d14947f9931fdea29f0f8e3c; no voters: 87116910e3734e87b64cd77012c35417, f5ca2cdc8d61405080d925fa399fc98b
I20260811 17:23:13.228379 3707 raft_consensus.cc:2726] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 17:23:13.422013 3708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:13.421613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.129:55683)
0811 17:23:13.421845 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:13.421918 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 17:23:13.602145 3716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:13.601548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.130:45713)
0811 17:23:13.601890 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:13.601986 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":84,"threads_started":1}
I20260811 17:23:13.810671 3717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:13.810127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.131:50413)
0811 17:23:13.810474 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:13.810546 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":82,"threads_started":1}
I20260811 17:23:14.423763 3720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:14.423058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.129:46651)
0811 17:23:14.423479 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:14.423550 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":93,"threads_started":1}
I20260811 17:23:14.603667 3721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:14.603137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.130:46961)
0811 17:23:14.603457 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:14.603530 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":86,"threads_started":1}
I20260811 17:23:14.812263 3722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:14.811688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.131:44823)
0811 17:23:14.812065 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:14.812128 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":122,"threads_started":1}
I20260811 17:23:14.937353 3724 raft_consensus.cc:482] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:23:14.937556 3724 raft_consensus.cc:504] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } } peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } }
I20260811 17:23:14.938115 3724 leader_election.cc:288] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 87116910e3734e87b64cd77012c35417 (127.9.157.130:40213), f5ca2cdc8d61405080d925fa399fc98b (127.9.157.131:36241)
W20260811 17:23:14.939209 3104 leader_election.cc:341] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 87116910e3734e87b64cd77012c35417 (127.9.157.130:40213): Not found: Tablet not found: f74b4708125d46ee9b2676097b48f984
W20260811 17:23:14.939350 3110 leader_election.cc:341] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f5ca2cdc8d61405080d925fa399fc98b (127.9.157.131:36241): Not found: Tablet not found: f74b4708125d46ee9b2676097b48f984
I20260811 17:23:14.939488 3110 leader_election.cc:302] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b46d01b7d14947f9931fdea29f0f8e3c; no voters: 87116910e3734e87b64cd77012c35417, f5ca2cdc8d61405080d925fa399fc98b
I20260811 17:23:14.939745 3724 raft_consensus.cc:2726] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 17:23:15.425053 3726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:15.424558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.129:39819)
0811 17:23:15.424866 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:15.424928 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":79,"threads_started":1}
I20260811 17:23:15.605136 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:15.604648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.130:47663)
0811 17:23:15.604954 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:15.605020 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":85,"threads_started":1}
I20260811 17:23:15.814062 3728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:15.813331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.131:53349)
0811 17:23:15.813714 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:15.813780 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":143,"threads_started":1}
I20260811 17:23:16.426779 3732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:16.426232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.129:50553)
0811 17:23:16.426590 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:16.426671 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":147,"threads_started":1}
I20260811 17:23:16.606748 3733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:16.606222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.130:35065)
0811 17:23:16.606565 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:16.606641 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":93,"threads_started":1}
I20260811 17:23:16.715260 3735 raft_consensus.cc:482] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:23:16.715405 3735 raft_consensus.cc:504] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } } peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } }
I20260811 17:23:16.715961 3735 leader_election.cc:288] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 87116910e3734e87b64cd77012c35417 (127.9.157.130:40213), f5ca2cdc8d61405080d925fa399fc98b (127.9.157.131:36241)
W20260811 17:23:16.716802 3104 leader_election.cc:341] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 87116910e3734e87b64cd77012c35417 (127.9.157.130:40213): Not found: Tablet not found: f74b4708125d46ee9b2676097b48f984
W20260811 17:23:16.716897 3110 leader_election.cc:341] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f5ca2cdc8d61405080d925fa399fc98b (127.9.157.131:36241): Not found: Tablet not found: f74b4708125d46ee9b2676097b48f984
I20260811 17:23:16.717021 3110 leader_election.cc:302] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b46d01b7d14947f9931fdea29f0f8e3c; no voters: 87116910e3734e87b64cd77012c35417, f5ca2cdc8d61405080d925fa399fc98b
I20260811 17:23:16.717211 3735 raft_consensus.cc:2726] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 17:23:16.815693 3736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:16.815175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.131:59537)
0811 17:23:16.815480 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:16.815551 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":88,"threads_started":1}
I20260811 17:23:17.428294 3738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:17.427781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.129:33985)
0811 17:23:17.428105 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:17.428184 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":84,"threads_started":1}
I20260811 17:23:17.608410 3739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:17.607890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.130:50101)
0811 17:23:17.608217 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:17.608289 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":83,"threads_started":1}
I20260811 17:23:17.817291 3741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:17.816765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.131:55899)
0811 17:23:17.817111 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:17.817185 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":93,"threads_started":1}
I20260811 17:23:18.429768 3742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:18.429308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.129:36871)
0811 17:23:18.429600 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:18.429666 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":95,"threads_started":1}
I20260811 17:23:18.609792 3743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:18.609245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.130:41275)
0811 17:23:18.609633 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:18.609690 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":140,"threads_started":1}
I20260811 17:23:18.614013 3745 raft_consensus.cc:482] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:23:18.614161 3745 raft_consensus.cc:504] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } } peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } }
I20260811 17:23:18.614566 3745 leader_election.cc:288] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 87116910e3734e87b64cd77012c35417 (127.9.157.130:40213), f5ca2cdc8d61405080d925fa399fc98b (127.9.157.131:36241)
W20260811 17:23:18.615438 3104 leader_election.cc:341] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 87116910e3734e87b64cd77012c35417 (127.9.157.130:40213): Not found: Tablet not found: f74b4708125d46ee9b2676097b48f984
W20260811 17:23:18.615541 3110 leader_election.cc:341] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f5ca2cdc8d61405080d925fa399fc98b (127.9.157.131:36241): Not found: Tablet not found: f74b4708125d46ee9b2676097b48f984
I20260811 17:23:18.615684 3110 leader_election.cc:302] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b46d01b7d14947f9931fdea29f0f8e3c; no voters: 87116910e3734e87b64cd77012c35417, f5ca2cdc8d61405080d925fa399fc98b
I20260811 17:23:18.615861 3745 raft_consensus.cc:2726] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 17:23:18.819013 3747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:18.818389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.131:44453)
0811 17:23:18.818789 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:18.818861 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":89,"threads_started":1}
I20260811 17:23:19.431064 3749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:19.430622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.129:59811)
0811 17:23:19.430894 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:19.430961 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":85,"threads_started":1}
I20260811 17:23:19.611212 3750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:19.610754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.130:53189)
0811 17:23:19.611046 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:19.611108 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":83,"threads_started":1}
I20260811 17:23:19.820568 3751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:19.820056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.131:34415)
0811 17:23:19.820403 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:19.820465 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":105,"threads_started":1}
I20260811 17:23:20.432449 3752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:20.431944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.129:42389)
0811 17:23:20.432256 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:20.432344 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":130,"threads_started":1}
I20260811 17:23:20.612663 3754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:20.612171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.130:50589)
0811 17:23:20.612477 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:20.612543 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":90,"threads_started":1}
I20260811 17:23:20.822113 3755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:20.821548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.131:32969)
0811 17:23:20.821902 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:20.821996 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":127,"threads_started":1}
I20260811 17:23:21.434013 3756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:21.433495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.129:53061)
0811 17:23:21.433810 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:21.433892 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":127,"threads_started":1}
I20260811 17:23:21.614095 3758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:21.613629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.130:40971)
0811 17:23:21.613936 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:21.613995 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":103,"threads_started":1}
I20260811 17:23:21.823913 3759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:21.823309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.131:45535)
0811 17:23:21.823693 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:21.823778 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":136,"threads_started":1}
I20260811 17:23:22.435411 3763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:22.434924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.129:38667)
0811 17:23:22.435240 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:22.435304 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":117,"threads_started":1}
I20260811 17:23:22.615543 3764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:22.615021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.130:47257)
0811 17:23:22.615368 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:22.615433 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":108,"threads_started":1}
I20260811 17:23:22.825428 3765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:22.824911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.131:46357)
0811 17:23:22.825244 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:22.825313 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":97,"threads_started":1}
I20260811 17:23:23.436767 3766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:23.436294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.129:38899)
0811 17:23:23.436595 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:23.436662 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":88,"threads_started":1}
I20260811 17:23:23.617023 3767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:23.616529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.130:53837)
0811 17:23:23.616842 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:23.616911 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":87,"threads_started":1}
I20260811 17:23:23.826798 3769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:23.826292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.131:53871)
0811 17:23:23.826609 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:23.826668 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":99,"threads_started":1}
I20260811 17:23:24.438249 3772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:24.437725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.129:36613)
0811 17:23:24.438066 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:24.438136 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":124,"threads_started":1}
I20260811 17:23:24.618444 3773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:24.617962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.130:53613)
0811 17:23:24.618259 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:24.618328 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":114,"threads_started":1}
I20260811 17:23:24.828384 3774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:24.827898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.131:47129)
0811 17:23:24.828194 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:24.828264 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":86,"threads_started":1}
I20260811 17:23:25.439960 3775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:25.439390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.129:49277)
0811 17:23:25.439772 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:25.439851 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":90,"threads_started":1}
I20260811 17:23:25.619989 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:25.619364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.130:57391)
0811 17:23:25.619803 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:25.619876 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":95,"threads_started":1}
I20260811 17:23:25.829993 3777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:25.829388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.131:39325)
0811 17:23:25.829740 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:25.829811 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":94,"threads_started":1}
I20260811 17:23:26.153577 3778 raft_consensus.cc:482] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:23:26.153766 3778 raft_consensus.cc:504] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87116910e3734e87b64cd77012c35417" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 40213 } } peers { permanent_uuid: "f5ca2cdc8d61405080d925fa399fc98b" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36241 } } peers { permanent_uuid: "b46d01b7d14947f9931fdea29f0f8e3c" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 43025 } }
I20260811 17:23:26.154498 3778 leader_election.cc:288] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 87116910e3734e87b64cd77012c35417 (127.9.157.130:40213), f5ca2cdc8d61405080d925fa399fc98b (127.9.157.131:36241)
W20260811 17:23:26.155187 3110 leader_election.cc:341] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f5ca2cdc8d61405080d925fa399fc98b (127.9.157.131:36241): Not found: Tablet not found: f74b4708125d46ee9b2676097b48f984
W20260811 17:23:26.155313 3104 leader_election.cc:341] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 87116910e3734e87b64cd77012c35417 (127.9.157.130:40213): Not found: Tablet not found: f74b4708125d46ee9b2676097b48f984
I20260811 17:23:26.155417 3104 leader_election.cc:302] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b46d01b7d14947f9931fdea29f0f8e3c; no voters: 87116910e3734e87b64cd77012c35417, f5ca2cdc8d61405080d925fa399fc98b
I20260811 17:23:26.155611 3778 raft_consensus.cc:2726] T f74b4708125d46ee9b2676097b48f984 P b46d01b7d14947f9931fdea29f0f8e3c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 17:23:26.441758 3779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:26.441185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.129:34997)
0811 17:23:26.441583 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:26.441649 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":120,"threads_started":1}
I20260811 17:23:26.621495 3780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:26.621040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.130:44563)
0811 17:23:26.621334 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:26.621394 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":82,"threads_started":1}
I20260811 17:23:26.809692 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 3031
I20260811 17:23:26.827072 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 3289
I20260811 17:23:26.833289 3781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:26.831528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:43235 (local address 127.9.157.131:39821)
0811 17:23:26.833116 (+ 1588us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:26.833183 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.190:43235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1498,"thread_start_us":1238,"threads_started":1}
I20260811 17:23:26.839432 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 3492
2026-08-11T17:23:26Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (60014 ms)
[----------] 30 tests from SecurityITest (262930 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:23:26 dist-test-slave-mjjr krb5kdc[3785](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:23:26 dist-test-slave-mjjr krb5kdc[3785](info): set up 2 sockets
Aug 11 17:23:26 dist-test-slave-mjjr krb5kdc[3785](info): commencing operation
krb5kdc: starting...
W20260811 17:23:28.122799 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:23:28 dist-test-slave-mjjr krb5kdc[3785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:23:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:23:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:23:28.263080 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:35855
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:38015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:35855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:23:28.386031 3802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:28.386379 3802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:28.386423 3802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:28.390728 3802 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:23:28.390810 3802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:28.390831 3802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:28.390848 3802 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:23:28.390864 3802 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:23:28.396667 3802 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:38015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:35855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:35855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:28.397853 3802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:28.398922 3802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:28.408596 3811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:28.408596 3816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:28.408596 3812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:28.408660 3802 server_base.cc:1034] running on GCE node
I20260811 17:23:28.409565 3802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:28.410846 3802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:28.412052 3802 hybrid_clock.cc:648] HybridClock initialized: now 1786469008412035 us; error 33 us; skew 500 ppm
Aug 11 17:23:28 dist-test-slave-mjjr krb5kdc[3785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469008, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:23:28.415062 3802 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:23:28.416173 3802 webserver.cc:460] Webserver started at http://127.9.157.190:41763/ using document root <none> and password file <none>
I20260811 17:23:28.416740 3802 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:28.416810 3802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:28.417006 3802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:28.419287 3802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "bc4c0abb62e8447e9d16d99f1c5edf07"
format_stamp: "Formatted at 2026-08-11 17:23:28 on dist-test-slave-mjjr"
server_key: "84d06fd38a6ac2035bb8a3d3d3f837ed"
server_key_iv: "7ad596aefca5c2256a330d0d50e2881e"
server_key_version: "encryptionkey@0"
I20260811 17:23:28.419941 3802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "bc4c0abb62e8447e9d16d99f1c5edf07"
format_stamp: "Formatted at 2026-08-11 17:23:28 on dist-test-slave-mjjr"
server_key: "84d06fd38a6ac2035bb8a3d3d3f837ed"
server_key_iv: "7ad596aefca5c2256a330d0d50e2881e"
server_key_version: "encryptionkey@0"
I20260811 17:23:28.424114 3802 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 17:23:28.426827 3826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:28.428267 3802 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:23:28.428390 3802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "bc4c0abb62e8447e9d16d99f1c5edf07"
format_stamp: "Formatted at 2026-08-11 17:23:28 on dist-test-slave-mjjr"
server_key: "84d06fd38a6ac2035bb8a3d3d3f837ed"
server_key_iv: "7ad596aefca5c2256a330d0d50e2881e"
server_key_version: "encryptionkey@0"
I20260811 17:23:28.428495 3802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:28.454988 3802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:28.458446 3802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:28.458633 3802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:28.466703 3802 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:35855
I20260811 17:23:28.466876 3900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:35855 every 8 connection(s)
I20260811 17:23:28.467861 3802 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:23:28.470942 3902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:28.472093 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 3802
I20260811 17:23:28.472235 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:23:28.472561 9846 external_mini_cluster.cc:1428] Setting key aefa45f9a040e829719289f9f9d21dc7
I20260811 17:23:28.477872 3902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc4c0abb62e8447e9d16d99f1c5edf07: Bootstrap starting.
I20260811 17:23:28.480595 3902 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc4c0abb62e8447e9d16d99f1c5edf07: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:28.481565 3902 log.cc:824] T 00000000000000000000000000000000 P bc4c0abb62e8447e9d16d99f1c5edf07: Log is configured to *not* fsync() on all Append() calls
Aug 11 17:23:28 dist-test-slave-mjjr krb5kdc[3785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:23:28.484460 3902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc4c0abb62e8447e9d16d99f1c5edf07: No bootstrap required, opened a new log
I20260811 17:23:28.488518 3905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:28.474096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60658 (local address 127.9.157.190:35855)
0811 17:23:28.474593 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:28.474609 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:28.474647 (+ 38us) server_negotiation.cc:407] Connection header received
0811 17:23:28.475324 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:28.475344 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:28.475757 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:28.476111 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:28.476936 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:28.477939 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:28.478819 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:28.479304 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:28.479483 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:28.482643 (+ 3160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:28.482654 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:28.482678 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:23:28.482724 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:28.485023 (+ 2299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:28.485503 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:28.485510 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:28.485517 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:28.485669 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:28.485946 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:28.485949 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:28.485951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:28.486543 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:28.486792 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:28.487016 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 17:23:28.487421 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":175,"threads_started":1}
I20260811 17:23:28.489630 3902 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc4c0abb62e8447e9d16d99f1c5edf07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4c0abb62e8447e9d16d99f1c5edf07" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 35855 } }
I20260811 17:23:28.489940 3902 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc4c0abb62e8447e9d16d99f1c5edf07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:28.490039 3902 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc4c0abb62e8447e9d16d99f1c5edf07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc4c0abb62e8447e9d16d99f1c5edf07, State: Initialized, Role: FOLLOWER
I20260811 17:23:28.490538 3902 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc4c0abb62e8447e9d16d99f1c5edf07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4c0abb62e8447e9d16d99f1c5edf07" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 35855 } }
I20260811 17:23:28.490679 3902 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc4c0abb62e8447e9d16d99f1c5edf07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:23:28.490772 3902 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc4c0abb62e8447e9d16d99f1c5edf07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:23:28.490873 3902 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc4c0abb62e8447e9d16d99f1c5edf07 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:28.492254 3902 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc4c0abb62e8447e9d16d99f1c5edf07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4c0abb62e8447e9d16d99f1c5edf07" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 35855 } }
I20260811 17:23:28.492641 3902 leader_election.cc:302] T 00000000000000000000000000000000 P bc4c0abb62e8447e9d16d99f1c5edf07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc4c0abb62e8447e9d16d99f1c5edf07; no voters:
I20260811 17:23:28.493052 3902 leader_election.cc:288] T 00000000000000000000000000000000 P bc4c0abb62e8447e9d16d99f1c5edf07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:23:28.493186 3908 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc4c0abb62e8447e9d16d99f1c5edf07 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:23:28.493448 3908 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc4c0abb62e8447e9d16d99f1c5edf07 [term 1 LEADER]: Becoming Leader. State: Replica: bc4c0abb62e8447e9d16d99f1c5edf07, State: Running, Role: LEADER
I20260811 17:23:28.493774 3908 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc4c0abb62e8447e9d16d99f1c5edf07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4c0abb62e8447e9d16d99f1c5edf07" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 35855 } }
I20260811 17:23:28.494489 3902 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc4c0abb62e8447e9d16d99f1c5edf07 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:23:28.495772 3912 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc4c0abb62e8447e9d16d99f1c5edf07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc4c0abb62e8447e9d16d99f1c5edf07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4c0abb62e8447e9d16d99f1c5edf07" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 35855 } } }
I20260811 17:23:28.495908 3912 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc4c0abb62e8447e9d16d99f1c5edf07 [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:28.495570 3913 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc4c0abb62e8447e9d16d99f1c5edf07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc4c0abb62e8447e9d16d99f1c5edf07. Latest consensus state: current_term: 1 leader_uuid: "bc4c0abb62e8447e9d16d99f1c5edf07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4c0abb62e8447e9d16d99f1c5edf07" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 35855 } } }
I20260811 17:23:28.496297 3913 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc4c0abb62e8447e9d16d99f1c5edf07 [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:28.496353 3921 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:23:28.499665 3921 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:23:28.506469 3921 catalog_manager.cc:1347] Generated new cluster ID: e73b660d62d843a5b54ec787541f4b12
I20260811 17:23:28.506597 3921 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:23:28.516932 3921 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:23:28.518054 3921 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:23:28.523367 3921 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc4c0abb62e8447e9d16d99f1c5edf07: Generated new TSK 0
I20260811 17:23:28.524195 3921 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
I20260811 17:23:28.563103 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:0
--local_ip_for_outbound_sockets=127.9.157.129
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:35855
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:38015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:23:28.712930 3936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:28.713274 3936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:28.713322 3936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:28.717782 3936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:28.717860 3936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:28.717923 3936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:23:28.723690 3936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:38015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:35855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:28.724978 3936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:28.726037 3936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:28.736838 3943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:28.736810 3942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:28.736922 3936 server_base.cc:1034] running on GCE node
W20260811 17:23:28.736814 3946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:28.737610 3936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:28.738323 3936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:28.739526 3936 hybrid_clock.cc:648] HybridClock initialized: now 1786469008739507 us; error 31 us; skew 500 ppm
Aug 11 17:23:28 dist-test-slave-mjjr krb5kdc[3785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469008, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:23:28.742766 3936 init.cc:377] Logged in from keytab as kudu/127.9.157.129@KRBTEST.COM (short username kudu)
I20260811 17:23:28.743871 3936 webserver.cc:460] Webserver started at http://127.9.157.129:33271/ using document root <none> and password file <none>
I20260811 17:23:28.744501 3936 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:28.744599 3936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:28.744850 3936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:28.747228 3936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-0/data/instance:
uuid: "fdc0067da1d44ca0af366ac05d4905ef"
format_stamp: "Formatted at 2026-08-11 17:23:28 on dist-test-slave-mjjr"
server_key: "3fc2d52ea9ee2d736945974e2a7e5f0e"
server_key_iv: "6ca87ee7a2e7ddc6865e198302b5eb78"
server_key_version: "encryptionkey@0"
I20260811 17:23:28.747942 3936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance:
uuid: "fdc0067da1d44ca0af366ac05d4905ef"
format_stamp: "Formatted at 2026-08-11 17:23:28 on dist-test-slave-mjjr"
server_key: "3fc2d52ea9ee2d736945974e2a7e5f0e"
server_key_iv: "6ca87ee7a2e7ddc6865e198302b5eb78"
server_key_version: "encryptionkey@0"
I20260811 17:23:28.752322 3936 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:23:28.755230 3961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:28.756795 3936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:23:28.756991 3936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "fdc0067da1d44ca0af366ac05d4905ef"
format_stamp: "Formatted at 2026-08-11 17:23:28 on dist-test-slave-mjjr"
server_key: "3fc2d52ea9ee2d736945974e2a7e5f0e"
server_key_iv: "6ca87ee7a2e7ddc6865e198302b5eb78"
server_key_version: "encryptionkey@0"
I20260811 17:23:28.757133 3936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:28.779075 3936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:28.782954 3936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:28.783179 3936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:28.783933 3936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:23:28.784904 3936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:23:28.784974 3936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:28.785050 3936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:23:28.785090 3936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:28.795181 3936 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:34109
I20260811 17:23:28.795217 4126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:34109 every 8 connection(s)
I20260811 17:23:28.796360 3936 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:23:28.803862 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 3936
I20260811 17:23:28.804029 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance
I20260811 17:23:28.804363 9846 external_mini_cluster.cc:1428] Setting key 15e8ff0483c40759436fbd6400547524
Aug 11 17:23:28 dist-test-slave-mjjr krb5kdc[3785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469008, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:23:28.813468 3905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:28.798423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:37687 (local address 127.9.157.190:35855)
0811 17:23:28.798613 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:28.798618 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:28.799841 (+ 1223us) server_negotiation.cc:407] Connection header received
0811 17:23:28.801384 (+ 1543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:28.801387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:28.801443 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:28.801528 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:28.802853 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:28.803403 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:28.805183 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:28.805463 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:28.805587 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:28.809314 (+ 3727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:28.809318 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:28.809320 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:23:28.809352 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:28.810904 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:28.811560 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:28.811563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:28.811565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:28.811737 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:28.812149 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:28.812153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:28.812155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:28.812383 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:28.812496 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:28.813123 (+ 627us) server_negotiation.cc:299] Negotiation successful
0811 17:23:28.813285 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 17:23:28.814450 4130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:28.798886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:35855 (local address 127.9.157.129:37687)
0811 17:23:28.799581 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:28.799633 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:28.801140 (+ 1507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:28.801715 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:28.801723 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:28.802140 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:23:28.802700 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:28.802715 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:28.803595 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:28.803600 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:28.805062 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:28.805074 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:28.805644 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:28.805649 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:28.805963 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:28.806821 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:23:28.806840 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:23:28.809060 (+ 2220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:23:28.811066 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:23:28.811073 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:23:28.811097 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:23:28.811427 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:23:28.811840 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:23:28.811844 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:23:28.811846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:23:28.812038 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:23:28.812502 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:23:28.812513 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:23:28.812843 (+ 330us) client_negotiation.cc:769] Sending connection context
0811 17:23:28.813058 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 17:23:28.813461 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":149,"threads_started":1}
I20260811 17:23:28.816067 4127 heartbeater.cc:347] Connected to a master server at 127.9.157.190:35855
WARNING: no policy specified for kudu/127.9.157.130@KRBTEST.COM; defaulting to no policy
I20260811 17:23:28.816471 4127 heartbeater.cc:464] Registering TS with master...
I20260811 17:23:28.817157 4127 heartbeater.cc:511] Master 127.9.157.190:35855 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.130@KRBTEST.COM" created.
I20260811 17:23:28.819720 3848 ts_manager.cc:194] Registered new tserver with Master: fdc0067da1d44ca0af366ac05d4905ef (127.9.157.129:34109)
I20260811 17:23:28.821612 3848 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.129@KRBTEST.COM'} at 127.9.157.129:37687
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
I20260811 17:23:28.839918 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:0
--local_ip_for_outbound_sockets=127.9.157.130
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:35855
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:38015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:23:28.965873 4134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:28.966197 4134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:28.966243 4134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:28.970598 4134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:28.970681 4134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:28.970745 4134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:23:28.976598 4134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:38015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:35855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:28.977871 4134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:28.979060 4134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:28.989785 4144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:28.989800 4141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:28.989861 4142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:28.990068 4134 server_base.cc:1034] running on GCE node
I20260811 17:23:28.990705 4134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:28.991475 4134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:28.992700 4134 hybrid_clock.cc:648] HybridClock initialized: now 1786469008992654 us; error 58 us; skew 500 ppm
Aug 11 17:23:28 dist-test-slave-mjjr krb5kdc[3785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469008, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:23:28.995997 4134 init.cc:377] Logged in from keytab as kudu/127.9.157.130@KRBTEST.COM (short username kudu)
I20260811 17:23:28.997102 4134 webserver.cc:460] Webserver started at http://127.9.157.130:34669/ using document root <none> and password file <none>
I20260811 17:23:28.997690 4134 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:28.997754 4134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:28.997939 4134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:29.000322 4134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-1/data/instance:
uuid: "522f9f44c542456993bf3390d882b85a"
format_stamp: "Formatted at 2026-08-11 17:23:28 on dist-test-slave-mjjr"
server_key: "f1bd3e059f05e6222ece1925f46582dc"
server_key_iv: "2512ac2cf3cd90edb773eb49537422af"
server_key_version: "encryptionkey@0"
I20260811 17:23:29.000965 4134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance:
uuid: "522f9f44c542456993bf3390d882b85a"
format_stamp: "Formatted at 2026-08-11 17:23:28 on dist-test-slave-mjjr"
server_key: "f1bd3e059f05e6222ece1925f46582dc"
server_key_iv: "2512ac2cf3cd90edb773eb49537422af"
server_key_version: "encryptionkey@0"
I20260811 17:23:29.005324 4134 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:23:29.008255 4157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:29.009630 4134 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:23:29.009783 4134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "522f9f44c542456993bf3390d882b85a"
format_stamp: "Formatted at 2026-08-11 17:23:28 on dist-test-slave-mjjr"
server_key: "f1bd3e059f05e6222ece1925f46582dc"
server_key_iv: "2512ac2cf3cd90edb773eb49537422af"
server_key_version: "encryptionkey@0"
I20260811 17:23:29.009909 4134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:29.025345 4134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:29.028940 4134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:29.029163 4134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:29.029816 4134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:23:29.030696 4134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:23:29.030748 4134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:29.030828 4134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:23:29.030866 4134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:29.040370 4134 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:46205
I20260811 17:23:29.040390 4330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:46205 every 8 connection(s)
I20260811 17:23:29.041354 4134 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:23:29.048683 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 4134
I20260811 17:23:29.048832 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance
I20260811 17:23:29.049175 9846 external_mini_cluster.cc:1428] Setting key db97142fb52fcc0804e4330fde4fa8f6
Aug 11 17:23:29 dist-test-slave-mjjr krb5kdc[3785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469008, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:23:29.057101 3905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:29.043345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:57255 (local address 127.9.157.190:35855)
0811 17:23:29.043513 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:29.043517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:29.044491 (+ 974us) server_negotiation.cc:407] Connection header received
0811 17:23:29.045959 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:29.045963 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:29.046024 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:29.046133 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:29.047408 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.048068 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.049761 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.049993 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.050104 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:29.053103 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:29.053107 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:29.053109 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:23:29.053138 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:29.054697 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:29.055299 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:29.055303 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:29.055305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:29.055402 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:29.055835 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:29.055839 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:29.055840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:29.056052 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:29.056167 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:29.056754 (+ 587us) server_negotiation.cc:299] Negotiation successful
0811 17:23:29.056915 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 17:23:29.058017 4334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:29.043798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:35855 (local address 127.9.157.130:57255)
0811 17:23:29.044312 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:29.044359 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:29.045721 (+ 1362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:29.046320 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:29.046328 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:29.046733 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:23:29.047243 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:29.047257 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.048210 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.048214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:29.049623 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:29.049635 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.050167 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.050172 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:29.050354 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:29.050989 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:23:29.051011 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:23:29.052889 (+ 1878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:23:29.054845 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:23:29.054852 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:23:29.054873 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:23:29.055185 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:23:29.055511 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:23:29.055516 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:23:29.055518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:23:29.055725 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:23:29.056167 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:23:29.056176 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:23:29.056485 (+ 309us) client_negotiation.cc:769] Sending connection context
0811 17:23:29.056692 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 17:23:29.057072 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":148,"threads_started":1}
I20260811 17:23:29.059381 4331 heartbeater.cc:347] Connected to a master server at 127.9.157.190:35855
I20260811 17:23:29.059741 4331 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.9.157.131@KRBTEST.COM; defaulting to no policy
I20260811 17:23:29.060460 4331 heartbeater.cc:511] Master 127.9.157.190:35855 requested a full tablet report, sending...
I20260811 17:23:29.061810 3848 ts_manager.cc:194] Registered new tserver with Master: 522f9f44c542456993bf3390d882b85a (127.9.157.130:46205)
I20260811 17:23:29.062546 3848 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.130@KRBTEST.COM'} at 127.9.157.130:57255
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
I20260811 17:23:29.080736 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:0
--local_ip_for_outbound_sockets=127.9.157.131
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:35855
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:38015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:23:29.203011 4338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:29.203334 4338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:29.203389 4338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:29.207954 4338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:29.208033 4338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:29.208128 4338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:23:29.213997 4338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:38015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:35855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:29.215301 4338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:29.216516 4338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:29.227905 4354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:29.227905 4352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:29.227908 4351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:29.228302 4338 server_base.cc:1034] running on GCE node
I20260811 17:23:29.228794 4338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:29.229513 4338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:29.230719 4338 hybrid_clock.cc:648] HybridClock initialized: now 1786469009230698 us; error 32 us; skew 500 ppm
Aug 11 17:23:29 dist-test-slave-mjjr krb5kdc[3785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469009, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:23:29.234272 4338 init.cc:377] Logged in from keytab as kudu/127.9.157.131@KRBTEST.COM (short username kudu)
I20260811 17:23:29.235394 4338 webserver.cc:460] Webserver started at http://127.9.157.131:35895/ using document root <none> and password file <none>
I20260811 17:23:29.236114 4338 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:29.236217 4338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:29.236476 4338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:29.238739 4338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-2/data/instance:
uuid: "775ff2071ea542d8ba81c7e382c97a43"
format_stamp: "Formatted at 2026-08-11 17:23:29 on dist-test-slave-mjjr"
server_key: "5bc560633a18dbba2cc650af0d2ee169"
server_key_iv: "9a92fcc72daac2f13fe07c80a07fc2ce"
server_key_version: "encryptionkey@0"
I20260811 17:23:29.239413 4338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance:
uuid: "775ff2071ea542d8ba81c7e382c97a43"
format_stamp: "Formatted at 2026-08-11 17:23:29 on dist-test-slave-mjjr"
server_key: "5bc560633a18dbba2cc650af0d2ee169"
server_key_iv: "9a92fcc72daac2f13fe07c80a07fc2ce"
server_key_version: "encryptionkey@0"
I20260811 17:23:29.243896 4338 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 17:23:29.246726 4373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:29.248355 4338 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:23:29.248512 4338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "775ff2071ea542d8ba81c7e382c97a43"
format_stamp: "Formatted at 2026-08-11 17:23:29 on dist-test-slave-mjjr"
server_key: "5bc560633a18dbba2cc650af0d2ee169"
server_key_iv: "9a92fcc72daac2f13fe07c80a07fc2ce"
server_key_version: "encryptionkey@0"
I20260811 17:23:29.248646 4338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:29.259243 4338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:29.262842 4338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:29.263089 4338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:29.263857 4338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:23:29.264806 4338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:23:29.264886 4338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:29.264959 4338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:23:29.264998 4338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:29.274896 4338 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:44281
I20260811 17:23:29.274915 4555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:44281 every 8 connection(s)
I20260811 17:23:29.276057 4338 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:23:29.279260 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 4338
I20260811 17:23:29.279443 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance
I20260811 17:23:29.279821 9846 external_mini_cluster.cc:1428] Setting key 71ef4a491032f19006ec7a852704cb43
Aug 11 17:23:29 dist-test-slave-mjjr krb5kdc[3785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469009, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:23:29.291733 3905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:29.278304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:38783 (local address 127.9.157.190:35855)
0811 17:23:29.278483 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:29.278487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:29.279464 (+ 977us) server_negotiation.cc:407] Connection header received
0811 17:23:29.281434 (+ 1970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:29.281438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:29.281503 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:29.281602 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:29.283048 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.283596 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.284937 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.285179 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.285300 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:29.288135 (+ 2835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:29.288139 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:29.288141 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:23:29.288171 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:29.289461 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:29.290046 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:29.290049 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:29.290050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:29.290131 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:29.290498 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:29.290502 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:29.290504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:29.290707 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:29.290817 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:29.291385 (+ 568us) server_negotiation.cc:299] Negotiation successful
0811 17:23:29.291523 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 17:23:29.292703 4559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:29.278709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:35855 (local address 127.9.157.131:38783)
0811 17:23:29.279282 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:29.279341 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:29.281157 (+ 1816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:29.281804 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:29.281812 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:29.282332 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:23:29.282886 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:29.282901 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.283775 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.283779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:29.284778 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:29.284791 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.285360 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.285364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:29.285570 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:29.286200 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:23:29.286218 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:23:29.287911 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:23:29.289604 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:23:29.289611 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:23:29.289631 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:23:29.289924 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:23:29.290235 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:23:29.290238 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:23:29.290240 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:23:29.290400 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:23:29.290818 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:23:29.290827 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:23:29.291119 (+ 292us) client_negotiation.cc:769] Sending connection context
0811 17:23:29.291333 (+ 214us) client_negotiation.cc:240] Negotiation successful
0811 17:23:29.291769 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":183,"threads_started":1}
I20260811 17:23:29.294052 4556 heartbeater.cc:347] Connected to a master server at 127.9.157.190:35855
I20260811 17:23:29.294348 4556 heartbeater.cc:464] Registering TS with master...
I20260811 17:23:29.294967 4556 heartbeater.cc:511] Master 127.9.157.190:35855 requested a full tablet report, sending...
I20260811 17:23:29.296123 3848 ts_manager.cc:194] Registered new tserver with Master: 775ff2071ea542d8ba81c7e382c97a43 (127.9.157.131:44281)
I20260811 17:23:29.296829 3848 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.131@KRBTEST.COM'} at 127.9.157.131:38783
I20260811 17:23:29.306603 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:23:29.316484 3905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:29.308895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60668 (local address 127.9.157.190:35855)
0811 17:23:29.309063 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:29.309067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:29.309188 (+ 121us) server_negotiation.cc:407] Connection header received
0811 17:23:29.309295 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:29.309297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:29.309346 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:29.309425 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:29.310176 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.310704 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.311547 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.311784 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.311911 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:29.313282 (+ 1371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:29.313286 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:29.313288 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:23:29.313314 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:29.314816 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:29.315314 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:29.315318 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:29.315319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:29.315399 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:29.315682 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:29.315688 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:29.315690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:29.315937 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:29.316050 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:29.316257 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 17:23:29.316355 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 17:23:29.319700 3848 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60668:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:23:29.322835 3848 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:23:29.339344 4583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:29.333535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47840 (local address 127.9.157.131:44281)
0811 17:23:29.334509 (+ 974us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:29.334522 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:29.334545 (+ 23us) server_negotiation.cc:407] Connection header received
0811 17:23:29.334619 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:29.334626 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:29.334818 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:29.334970 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:29.335298 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.336160 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.337784 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.338537 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.338650 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:29.338818 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:29.339052 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 17:23:29.339170 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":860,"thread_start_us":135,"threads_started":1}
I20260811 17:23:29.339334 4578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:29.333343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:44281 (local address 127.0.0.1:47840)
0811 17:23:29.334114 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:29.334129 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:29.334221 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:29.335110 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:29.335112 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:29.335127 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:29.335187 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:29.335191 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.336298 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.336303 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:29.337658 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:29.337669 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.338676 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.338680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:29.338804 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:29.338852 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 17:23:29.339001 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 17:23:29.339150 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"mutex_wait_us":44,"thread_start_us":109,"threads_started":1}
I20260811 17:23:29.339490 4574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:29.332449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:34109 (local address 127.0.0.1:60278)
0811 17:23:29.332878 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:29.332943 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:29.333098 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:29.334557 (+ 1459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:29.334563 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:29.334603 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:29.334794 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:29.334801 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.336453 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.336455 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:29.338074 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:29.338086 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.339078 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.339081 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:29.339162 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:29.339184 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 17:23:29.339232 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 17:23:29.339291 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":213,"spinlock_wait_cycles":4480,"thread_start_us":132,"threads_started":1}
I20260811 17:23:29.340080 4582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:29.332531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:46205 (local address 127.0.0.1:57680)
0811 17:23:29.333941 (+ 1410us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:29.333956 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:29.334061 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:29.334554 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:29.334563 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:29.334608 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:29.334804 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:29.334812 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.336584 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.336586 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:29.338074 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:29.338086 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.339346 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.339350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:29.339737 (+ 387us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:29.339764 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 17:23:29.339846 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 17:23:29.339925 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1332,"mutex_wait_us":244,"spinlock_wait_cycles":13056,"thread_start_us":118,"threads_started":1}
I20260811 17:23:29.340160 4581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:29.333120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57680 (local address 127.9.157.130:46205)
0811 17:23:29.333659 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:29.333674 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:29.334162 (+ 488us) server_negotiation.cc:407] Connection header received
0811 17:23:29.334236 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:29.334240 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:29.334383 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:29.334550 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:29.334963 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.336271 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.338216 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.339035 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.339141 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:29.339275 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:29.339892 (+ 617us) server_negotiation.cc:299] Negotiation successful
0811 17:23:29.340012 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":166,"threads_started":1}
I20260811 17:23:29.340989 4585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:29.333703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60278 (local address 127.9.157.129:34109)
0811 17:23:29.334027 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:29.334034 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:29.334061 (+ 27us) server_negotiation.cc:407] Connection header received
0811 17:23:29.334146 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:29.334153 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:29.334379 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:29.334969 (+ 590us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:29.335117 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.336276 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.338237 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.338943 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.339621 (+ 678us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:29.340756 (+ 1135us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:29.340800 (+ 44us) server_negotiation.cc:299] Negotiation successful
0811 17:23:29.340874 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":113,"threads_started":1}
I20260811 17:23:29.342659 4457 tablet_service.cc:1467] Processing CreateTablet for tablet c79e98d2740347feb97419e94bce0ad7 (DEFAULT_TABLE table=test-table [id=54019f982264435e947f1fda42fd87fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:23:29.343850 4457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c79e98d2740347feb97419e94bce0ad7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:29.343693 4039 tablet_service.cc:1467] Processing CreateTablet for tablet c79e98d2740347feb97419e94bce0ad7 (DEFAULT_TABLE table=test-table [id=54019f982264435e947f1fda42fd87fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:23:29.343699 4229 tablet_service.cc:1467] Processing CreateTablet for tablet c79e98d2740347feb97419e94bce0ad7 (DEFAULT_TABLE table=test-table [id=54019f982264435e947f1fda42fd87fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:23:29.344808 4229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c79e98d2740347feb97419e94bce0ad7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:29.344808 4039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c79e98d2740347feb97419e94bce0ad7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:29.350940 4586 tablet_bootstrap.cc:492] T c79e98d2740347feb97419e94bce0ad7 P 775ff2071ea542d8ba81c7e382c97a43: Bootstrap starting.
I20260811 17:23:29.353001 4588 tablet_bootstrap.cc:492] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a: Bootstrap starting.
I20260811 17:23:29.354022 4586 tablet_bootstrap.cc:654] T c79e98d2740347feb97419e94bce0ad7 P 775ff2071ea542d8ba81c7e382c97a43: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:29.355031 4588 tablet_bootstrap.cc:654] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:29.355233 4586 log.cc:824] T c79e98d2740347feb97419e94bce0ad7 P 775ff2071ea542d8ba81c7e382c97a43: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:29.356081 4587 tablet_bootstrap.cc:492] T c79e98d2740347feb97419e94bce0ad7 P fdc0067da1d44ca0af366ac05d4905ef: Bootstrap starting.
I20260811 17:23:29.356264 4588 log.cc:824] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:29.357616 4586 tablet_bootstrap.cc:492] T c79e98d2740347feb97419e94bce0ad7 P 775ff2071ea542d8ba81c7e382c97a43: No bootstrap required, opened a new log
I20260811 17:23:29.357851 4586 ts_tablet_manager.cc:1397] T c79e98d2740347feb97419e94bce0ad7 P 775ff2071ea542d8ba81c7e382c97a43: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 17:23:29.358330 4587 tablet_bootstrap.cc:654] T c79e98d2740347feb97419e94bce0ad7 P fdc0067da1d44ca0af366ac05d4905ef: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:29.358417 4588 tablet_bootstrap.cc:492] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a: No bootstrap required, opened a new log
I20260811 17:23:29.358654 4588 ts_tablet_manager.cc:1397] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260811 17:23:29.359213 4587 log.cc:824] T c79e98d2740347feb97419e94bce0ad7 P fdc0067da1d44ca0af366ac05d4905ef: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:29.361335 4587 tablet_bootstrap.cc:492] T c79e98d2740347feb97419e94bce0ad7 P fdc0067da1d44ca0af366ac05d4905ef: No bootstrap required, opened a new log
I20260811 17:23:29.361549 4587 ts_tablet_manager.cc:1397] T c79e98d2740347feb97419e94bce0ad7 P fdc0067da1d44ca0af366ac05d4905ef: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 17:23:29.362111 4586 raft_consensus.cc:348] T c79e98d2740347feb97419e94bce0ad7 P 775ff2071ea542d8ba81c7e382c97a43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0067da1d44ca0af366ac05d4905ef" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 34109 } } peers { permanent_uuid: "522f9f44c542456993bf3390d882b85a" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 46205 } } peers { permanent_uuid: "775ff2071ea542d8ba81c7e382c97a43" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44281 } }
I20260811 17:23:29.362429 4586 raft_consensus.cc:374] T c79e98d2740347feb97419e94bce0ad7 P 775ff2071ea542d8ba81c7e382c97a43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:29.362521 4586 raft_consensus.cc:729] T c79e98d2740347feb97419e94bce0ad7 P 775ff2071ea542d8ba81c7e382c97a43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 775ff2071ea542d8ba81c7e382c97a43, State: Initialized, Role: FOLLOWER
I20260811 17:23:29.362566 4588 raft_consensus.cc:348] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0067da1d44ca0af366ac05d4905ef" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 34109 } } peers { permanent_uuid: "522f9f44c542456993bf3390d882b85a" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 46205 } } peers { permanent_uuid: "775ff2071ea542d8ba81c7e382c97a43" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44281 } }
I20260811 17:23:29.362859 4588 raft_consensus.cc:374] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:29.362921 4588 raft_consensus.cc:729] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522f9f44c542456993bf3390d882b85a, State: Initialized, Role: FOLLOWER
I20260811 17:23:29.363052 4586 consensus_queue.cc:260] T c79e98d2740347feb97419e94bce0ad7 P 775ff2071ea542d8ba81c7e382c97a43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0067da1d44ca0af366ac05d4905ef" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 34109 } } peers { permanent_uuid: "522f9f44c542456993bf3390d882b85a" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 46205 } } peers { permanent_uuid: "775ff2071ea542d8ba81c7e382c97a43" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44281 } }
I20260811 17:23:29.363335 4588 consensus_queue.cc:260] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0067da1d44ca0af366ac05d4905ef" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 34109 } } peers { permanent_uuid: "522f9f44c542456993bf3390d882b85a" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 46205 } } peers { permanent_uuid: "775ff2071ea542d8ba81c7e382c97a43" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44281 } }
I20260811 17:23:29.363901 4556 heartbeater.cc:503] Master 127.9.157.190:35855 was elected leader, sending a full tablet report...
I20260811 17:23:29.364231 4586 ts_tablet_manager.cc:1428] T c79e98d2740347feb97419e94bce0ad7 P 775ff2071ea542d8ba81c7e382c97a43: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260811 17:23:29.364275 4588 ts_tablet_manager.cc:1428] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a: Time spent starting tablet: real 0.006s user 0.000s sys 0.005s
I20260811 17:23:29.364588 4331 heartbeater.cc:503] Master 127.9.157.190:35855 was elected leader, sending a full tablet report...
I20260811 17:23:29.366575 4587 raft_consensus.cc:348] T c79e98d2740347feb97419e94bce0ad7 P fdc0067da1d44ca0af366ac05d4905ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0067da1d44ca0af366ac05d4905ef" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 34109 } } peers { permanent_uuid: "522f9f44c542456993bf3390d882b85a" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 46205 } } peers { permanent_uuid: "775ff2071ea542d8ba81c7e382c97a43" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44281 } }
I20260811 17:23:29.366921 4587 raft_consensus.cc:374] T c79e98d2740347feb97419e94bce0ad7 P fdc0067da1d44ca0af366ac05d4905ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:29.367002 4587 raft_consensus.cc:729] T c79e98d2740347feb97419e94bce0ad7 P fdc0067da1d44ca0af366ac05d4905ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdc0067da1d44ca0af366ac05d4905ef, State: Initialized, Role: FOLLOWER
I20260811 17:23:29.367470 4587 consensus_queue.cc:260] T c79e98d2740347feb97419e94bce0ad7 P fdc0067da1d44ca0af366ac05d4905ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0067da1d44ca0af366ac05d4905ef" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 34109 } } peers { permanent_uuid: "522f9f44c542456993bf3390d882b85a" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 46205 } } peers { permanent_uuid: "775ff2071ea542d8ba81c7e382c97a43" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44281 } }
I20260811 17:23:29.368307 4127 heartbeater.cc:503] Master 127.9.157.190:35855 was elected leader, sending a full tablet report...
I20260811 17:23:29.368754 4587 ts_tablet_manager.cc:1428] T c79e98d2740347feb97419e94bce0ad7 P fdc0067da1d44ca0af366ac05d4905ef: Time spent starting tablet: real 0.007s user 0.002s sys 0.005s
I20260811 17:23:29.515074 4594 raft_consensus.cc:482] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:23:29.515357 4594 raft_consensus.cc:504] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0067da1d44ca0af366ac05d4905ef" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 34109 } } peers { permanent_uuid: "522f9f44c542456993bf3390d882b85a" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 46205 } } peers { permanent_uuid: "775ff2071ea542d8ba81c7e382c97a43" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44281 } }
I20260811 17:23:29.516657 4594 leader_election.cc:288] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fdc0067da1d44ca0af366ac05d4905ef (127.9.157.129:34109), 775ff2071ea542d8ba81c7e382c97a43 (127.9.157.131:44281)
I20260811 17:23:29.521123 4585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:29.517202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:43283 (local address 127.9.157.129:34109)
0811 17:23:29.517390 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:29.517395 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:29.517415 (+ 20us) server_negotiation.cc:407] Connection header received
0811 17:23:29.517485 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:29.517488 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:29.517556 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:29.517656 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:29.518004 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.518670 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.519935 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.520479 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.520579 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:29.520625 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:29.520896 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 17:23:29.520979 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 17:23:29.521123 4334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:29.517055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:34109 (local address 127.9.157.130:43283)
0811 17:23:29.517209 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:29.517228 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:29.517356 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:29.517690 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:29.517693 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:29.517714 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:29.517866 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:29.517875 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.518792 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.518796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:29.519799 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:29.519809 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.520624 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.520628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:29.520746 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:29.520775 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 17:23:29.520866 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 17:23:29.520950 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":1}
I20260811 17:23:29.521340 4596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:29.517375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:44281 (local address 127.9.157.130:52407)
0811 17:23:29.517726 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:29.517740 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:29.517868 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:29.518104 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:29.518107 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:29.518125 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:29.518268 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:29.518275 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.519098 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.519101 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:29.520091 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:29.520102 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.520909 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.520913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:29.521009 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:29.521033 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 17:23:29.521111 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 17:23:29.521196 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":23,"thread_start_us":116,"threads_started":1}
I20260811 17:23:29.521365 4583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:29.517571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:52407 (local address 127.9.157.131:44281)
0811 17:23:29.517720 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:29.517724 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:29.517869 (+ 145us) server_negotiation.cc:407] Connection header received
0811 17:23:29.517953 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:29.517957 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:29.518007 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:29.518072 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:29.518383 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.518977 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.520237 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.520767 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.520917 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:29.520966 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:29.521147 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 17:23:29.521239 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 17:23:29.522003 4066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c79e98d2740347feb97419e94bce0ad7" candidate_uuid: "522f9f44c542456993bf3390d882b85a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdc0067da1d44ca0af366ac05d4905ef" is_pre_election: true
I20260811 17:23:29.522002 4486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c79e98d2740347feb97419e94bce0ad7" candidate_uuid: "522f9f44c542456993bf3390d882b85a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "775ff2071ea542d8ba81c7e382c97a43" is_pre_election: true
I20260811 17:23:29.522351 4066 raft_consensus.cc:2454] T c79e98d2740347feb97419e94bce0ad7 P fdc0067da1d44ca0af366ac05d4905ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522f9f44c542456993bf3390d882b85a in term 0.
I20260811 17:23:29.522360 4486 raft_consensus.cc:2454] T c79e98d2740347feb97419e94bce0ad7 P 775ff2071ea542d8ba81c7e382c97a43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522f9f44c542456993bf3390d882b85a in term 0.
I20260811 17:23:29.522902 4165 leader_election.cc:302] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522f9f44c542456993bf3390d882b85a, 775ff2071ea542d8ba81c7e382c97a43; no voters:
I20260811 17:23:29.523252 4594 raft_consensus.cc:2781] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:23:29.523345 4594 raft_consensus.cc:482] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:23:29.523377 4594 raft_consensus.cc:3037] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:29.524638 4594 raft_consensus.cc:504] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0067da1d44ca0af366ac05d4905ef" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 34109 } } peers { permanent_uuid: "522f9f44c542456993bf3390d882b85a" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 46205 } } peers { permanent_uuid: "775ff2071ea542d8ba81c7e382c97a43" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44281 } }
I20260811 17:23:29.525035 4594 leader_election.cc:288] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a [CANDIDATE]: Term 1 election: Requested vote from peers fdc0067da1d44ca0af366ac05d4905ef (127.9.157.129:34109), 775ff2071ea542d8ba81c7e382c97a43 (127.9.157.131:44281)
I20260811 17:23:29.525595 4066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c79e98d2740347feb97419e94bce0ad7" candidate_uuid: "522f9f44c542456993bf3390d882b85a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdc0067da1d44ca0af366ac05d4905ef"
I20260811 17:23:29.525614 4486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c79e98d2740347feb97419e94bce0ad7" candidate_uuid: "522f9f44c542456993bf3390d882b85a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "775ff2071ea542d8ba81c7e382c97a43"
I20260811 17:23:29.525759 4486 raft_consensus.cc:3037] T c79e98d2740347feb97419e94bce0ad7 P 775ff2071ea542d8ba81c7e382c97a43 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:29.525748 4066 raft_consensus.cc:3037] T c79e98d2740347feb97419e94bce0ad7 P fdc0067da1d44ca0af366ac05d4905ef [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:29.526964 4486 raft_consensus.cc:2454] T c79e98d2740347feb97419e94bce0ad7 P 775ff2071ea542d8ba81c7e382c97a43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522f9f44c542456993bf3390d882b85a in term 1.
I20260811 17:23:29.527034 4066 raft_consensus.cc:2454] T c79e98d2740347feb97419e94bce0ad7 P fdc0067da1d44ca0af366ac05d4905ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522f9f44c542456993bf3390d882b85a in term 1.
W20260811 17:23:29.527156 4558 tablet.cc:2369] T c79e98d2740347feb97419e94bce0ad7 P 775ff2071ea542d8ba81c7e382c97a43: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:23:29.527420 4165 leader_election.cc:302] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522f9f44c542456993bf3390d882b85a, fdc0067da1d44ca0af366ac05d4905ef; no voters:
I20260811 17:23:29.527842 4594 raft_consensus.cc:2781] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:23:29.528183 4594 raft_consensus.cc:686] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a [term 1 LEADER]: Becoming Leader. State: Replica: 522f9f44c542456993bf3390d882b85a, State: Running, Role: LEADER
I20260811 17:23:29.528554 4594 consensus_queue.cc:237] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0067da1d44ca0af366ac05d4905ef" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 34109 } } peers { permanent_uuid: "522f9f44c542456993bf3390d882b85a" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 46205 } } peers { permanent_uuid: "775ff2071ea542d8ba81c7e382c97a43" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44281 } }
I20260811 17:23:29.532303 3847 catalog_manager.cc:5495] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a reported cstate change: term changed from 0 to 1, leader changed from <none> to 522f9f44c542456993bf3390d882b85a (127.9.157.130). New cstate: current_term: 1 leader_uuid: "522f9f44c542456993bf3390d882b85a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdc0067da1d44ca0af366ac05d4905ef" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 34109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522f9f44c542456993bf3390d882b85a" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 46205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "775ff2071ea542d8ba81c7e382c97a43" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44281 } health_report { overall_health: UNKNOWN } } }
W20260811 17:23:29.542878 4333 tablet.cc:2369] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:23:29.547991 4128 tablet.cc:2369] T c79e98d2740347feb97419e94bce0ad7 P fdc0067da1d44ca0af366ac05d4905ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:23:29.583811 4581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:29.579854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57682 (local address 127.9.157.130:46205)
0811 17:23:29.580008 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:29.580013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:29.580029 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:23:29.580115 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:29.580117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:29.580177 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:29.580293 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:23:29.580579 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.581180 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.582084 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.582405 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.582491 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:29.582830 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:23:29.583323 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:23:29.583424 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:29.583515 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 17:23:29.583575 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 17:23:29.590596 4066 raft_consensus.cc:1261] T c79e98d2740347feb97419e94bce0ad7 P fdc0067da1d44ca0af366ac05d4905ef [term 1 FOLLOWER]: Refusing update from remote peer 522f9f44c542456993bf3390d882b85a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:23:29.590596 4486 raft_consensus.cc:1261] T c79e98d2740347feb97419e94bce0ad7 P 775ff2071ea542d8ba81c7e382c97a43 [term 1 FOLLOWER]: Refusing update from remote peer 522f9f44c542456993bf3390d882b85a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:23:29.591513 4594 consensus_queue.cc:1035] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdc0067da1d44ca0af366ac05d4905ef" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 34109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:23:29.591858 4597 consensus_queue.cc:1035] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a [LEADER]: Connected to new peer: Peer: permanent_uuid: "775ff2071ea542d8ba81c7e382c97a43" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:23:29.600423 4602 mvcc.cc:205] Tried to move back new op lower bound from 7317377063274221568 to 7317377063032033280. Current Snapshot: MvccSnapshot[applied={T|T < 7317377063274221568}]
I20260811 17:23:29.600893 4603 mvcc.cc:205] Tried to move back new op lower bound from 7317377063274221568 to 7317377063032033280. Current Snapshot: MvccSnapshot[applied={T|T < 7317377063274221568}]
I20260811 17:23:29.603988 4604 mvcc.cc:205] Tried to move back new op lower bound from 7317377063274221568 to 7317377063032033280. Current Snapshot: MvccSnapshot[applied={T|T < 7317377063274221568 or (T in {7317377063274221568})}]
I20260811 17:23:29.610508 3847 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:23:29.610693 3847 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:23:29.613926 3847 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bc4c0abb62e8447e9d16d99f1c5edf07: Sending DeleteTablet for 3 replicas of tablet c79e98d2740347feb97419e94bce0ad7
I20260811 17:23:29.615137 4229 tablet_service.cc:1514] Processing DeleteTablet for tablet c79e98d2740347feb97419e94bce0ad7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:23:29 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:57680
I20260811 17:23:29.615566 4039 tablet_service.cc:1514] Processing DeleteTablet for tablet c79e98d2740347feb97419e94bce0ad7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:23:29 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:60278
I20260811 17:23:29.616237 4616 tablet_replica.cc:330] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a: stopping tablet replica
I20260811 17:23:29.616374 4457 tablet_service.cc:1514] Processing DeleteTablet for tablet c79e98d2740347feb97419e94bce0ad7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:23:29 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:47840
I20260811 17:23:29.616645 4616 raft_consensus.cc:2229] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:23:29.617056 4617 tablet_replica.cc:330] T c79e98d2740347feb97419e94bce0ad7 P 775ff2071ea542d8ba81c7e382c97a43: stopping tablet replica
I20260811 17:23:29.617089 4616 raft_consensus.cc:2258] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:23:29.617496 4623 tablet_replica.cc:330] T c79e98d2740347feb97419e94bce0ad7 P fdc0067da1d44ca0af366ac05d4905ef: stopping tablet replica
I20260811 17:23:29.617833 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 3936
I20260811 17:23:29.617936 4617 raft_consensus.cc:2229] T c79e98d2740347feb97419e94bce0ad7 P 775ff2071ea542d8ba81c7e382c97a43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:23:29.618322 4617 raft_consensus.cc:2258] T c79e98d2740347feb97419e94bce0ad7 P 775ff2071ea542d8ba81c7e382c97a43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:23:29.624472 4617 ts_tablet_manager.cc:1905] T c79e98d2740347feb97419e94bce0ad7 P 775ff2071ea542d8ba81c7e382c97a43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:23:29.624781 4616 ts_tablet_manager.cc:1905] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:23:29.627781 4617 ts_tablet_manager.cc:1918] T c79e98d2740347feb97419e94bce0ad7 P 775ff2071ea542d8ba81c7e382c97a43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:23:29.627911 4617 log.cc:1196] T c79e98d2740347feb97419e94bce0ad7 P 775ff2071ea542d8ba81c7e382c97a43: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-2/wal/wals/c79e98d2740347feb97419e94bce0ad7
I20260811 17:23:29.628309 4617 ts_tablet_manager.cc:1939] T c79e98d2740347feb97419e94bce0ad7 P 775ff2071ea542d8ba81c7e382c97a43: Deleting consensus metadata
I20260811 17:23:29.628681 4616 ts_tablet_manager.cc:1918] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:23:29.628825 4616 log.cc:1196] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786468743878666-9846-0/minicluster-data/ts-1/wal/wals/c79e98d2740347feb97419e94bce0ad7
I20260811 17:23:29.629240 4616 ts_tablet_manager.cc:1939] T c79e98d2740347feb97419e94bce0ad7 P 522f9f44c542456993bf3390d882b85a: Deleting consensus metadata
I20260811 17:23:29.630918 3831 catalog_manager.cc:4841] TS 775ff2071ea542d8ba81c7e382c97a43 (127.9.157.131:44281): tablet c79e98d2740347feb97419e94bce0ad7 (table test-table [id=54019f982264435e947f1fda42fd87fa]) successfully deleted
I20260811 17:23:29.631045 3829 catalog_manager.cc:4841] TS 522f9f44c542456993bf3390d882b85a (127.9.157.130:46205): tablet c79e98d2740347feb97419e94bce0ad7 (table test-table [id=54019f982264435e947f1fda42fd87fa]) successfully deleted
W20260811 17:23:29.638216 3831 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.157.129:34109 (error 108)
I20260811 17:23:29.638304 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 4134
I20260811 17:23:29.645454 4582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:29.645095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:34109 (local address 127.0.0.1:60282)
0811 17:23:29.645237 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:29.645334 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.129:34109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 17:23:29.645711 3831 catalog_manager.cc:4584] TS fdc0067da1d44ca0af366ac05d4905ef (127.9.157.129:34109): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c79e98d2740347feb97419e94bce0ad7: Network error: Client connection negotiation failed: client connection to 127.9.157.129:34109: connect: Connection refused (error 111)
I20260811 17:23:29.653228 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 4338
I20260811 17:23:29.673293 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 3802
2026-08-11T17:23:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2838 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T17:23:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:23:29Z Disabled control of system clock
I20260811 17:23:29.719843 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:41205
--webserver_interface=127.9.157.190
--webserver_port=0
--builtin_ntp_servers=127.9.157.148:46095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:41205
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 17:23:29.845404 4630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:29.845738 4630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:29.845777 4630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:29.850042 4630 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:23:29.850129 4630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:29.850147 4630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:29.850164 4630 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:23:29.850179 4630 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:23:29.855237 4630 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:46095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:41205
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:41205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:29.856510 4630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:29.857969 4630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:29.867442 4636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:29.867493 4638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:29.867437 4641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:29.868312 4630 server_base.cc:1034] running on GCE node
I20260811 17:23:29.869254 4630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:29.871104 4630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:29.872340 4630 hybrid_clock.cc:648] HybridClock initialized: now 1786469009872314 us; error 48 us; skew 500 ppm
I20260811 17:23:29.878799 4630 webserver.cc:460] Webserver started at http://127.9.157.190:44587/ using document root <none> and password file <none>
I20260811 17:23:29.879563 4630 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:29.879714 4630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:29.879985 4630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:29.882323 4630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "3bf933b41ff74ce6bf4592f37de35187"
format_stamp: "Formatted at 2026-08-11 17:23:29 on dist-test-slave-mjjr"
server_key: "de5dbe34b5a47f83fb0e734b1342ef4a"
server_key_iv: "93d552ceb5e606cec9dc31091380930a"
server_key_version: "encryptionkey@0"
I20260811 17:23:29.882973 4630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "3bf933b41ff74ce6bf4592f37de35187"
format_stamp: "Formatted at 2026-08-11 17:23:29 on dist-test-slave-mjjr"
server_key: "de5dbe34b5a47f83fb0e734b1342ef4a"
server_key_iv: "93d552ceb5e606cec9dc31091380930a"
server_key_version: "encryptionkey@0"
I20260811 17:23:29.887516 4630 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 17:23:29.890575 4650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:29.892159 4630 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:23:29.892333 4630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "3bf933b41ff74ce6bf4592f37de35187"
format_stamp: "Formatted at 2026-08-11 17:23:29 on dist-test-slave-mjjr"
server_key: "de5dbe34b5a47f83fb0e734b1342ef4a"
server_key_iv: "93d552ceb5e606cec9dc31091380930a"
server_key_version: "encryptionkey@0"
I20260811 17:23:29.892472 4630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:29.920081 4630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:29.920964 4630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:29.921192 4630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:29.929122 4630 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:41205
I20260811 17:23:29.929150 4725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:41205 every 8 connection(s)
I20260811 17:23:29.930202 4630 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:23:29.933756 4727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:29.938134 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 4630
I20260811 17:23:29.938303 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:23:29.938640 9846 external_mini_cluster.cc:1428] Setting key f477941e9f8e55a9d12459613968c560
I20260811 17:23:29.940922 4727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bf933b41ff74ce6bf4592f37de35187: Bootstrap starting.
I20260811 17:23:29.943519 4727 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3bf933b41ff74ce6bf4592f37de35187: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:29.944464 4727 log.cc:824] T 00000000000000000000000000000000 P 3bf933b41ff74ce6bf4592f37de35187: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:29.947026 4727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bf933b41ff74ce6bf4592f37de35187: No bootstrap required, opened a new log
I20260811 17:23:29.949488 4736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:29.940378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57510 (local address 127.9.157.190:41205)
0811 17:23:29.941216 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:29.941235 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:29.941278 (+ 43us) server_negotiation.cc:407] Connection header received
0811 17:23:29.942012 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:29.942032 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:29.942487 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:29.942875 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:29.943405 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.944542 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.945792 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:29.946308 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:29.946510 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:29.947517 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:29.947523 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:29.947540 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:29.948049 (+ 509us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:29.948111 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:29.948134 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:29.948249 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:29.948411 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 17:23:29.948703 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":169,"threads_started":1}
I20260811 17:23:29.951476 4727 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3bf933b41ff74ce6bf4592f37de35187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bf933b41ff74ce6bf4592f37de35187" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41205 } }
I20260811 17:23:29.951788 4727 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3bf933b41ff74ce6bf4592f37de35187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:29.951871 4727 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3bf933b41ff74ce6bf4592f37de35187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bf933b41ff74ce6bf4592f37de35187, State: Initialized, Role: FOLLOWER
I20260811 17:23:29.952392 4727 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3bf933b41ff74ce6bf4592f37de35187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bf933b41ff74ce6bf4592f37de35187" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41205 } }
I20260811 17:23:29.952564 4727 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3bf933b41ff74ce6bf4592f37de35187 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:23:29.952638 4727 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3bf933b41ff74ce6bf4592f37de35187 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:23:29.952718 4727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3bf933b41ff74ce6bf4592f37de35187 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:29.953718 4727 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3bf933b41ff74ce6bf4592f37de35187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bf933b41ff74ce6bf4592f37de35187" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41205 } }
I20260811 17:23:29.954046 4727 leader_election.cc:302] T 00000000000000000000000000000000 P 3bf933b41ff74ce6bf4592f37de35187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3bf933b41ff74ce6bf4592f37de35187; no voters:
I20260811 17:23:29.954381 4727 leader_election.cc:288] T 00000000000000000000000000000000 P 3bf933b41ff74ce6bf4592f37de35187 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:23:29.954567 4739 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3bf933b41ff74ce6bf4592f37de35187 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:23:29.954855 4739 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3bf933b41ff74ce6bf4592f37de35187 [term 1 LEADER]: Becoming Leader. State: Replica: 3bf933b41ff74ce6bf4592f37de35187, State: Running, Role: LEADER
I20260811 17:23:29.955204 4739 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3bf933b41ff74ce6bf4592f37de35187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bf933b41ff74ce6bf4592f37de35187" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41205 } }
I20260811 17:23:29.955829 4727 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3bf933b41ff74ce6bf4592f37de35187 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:23:29.957105 4740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bf933b41ff74ce6bf4592f37de35187 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3bf933b41ff74ce6bf4592f37de35187" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bf933b41ff74ce6bf4592f37de35187" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41205 } } }
I20260811 17:23:29.957264 4740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bf933b41ff74ce6bf4592f37de35187 [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:29.957641 4742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bf933b41ff74ce6bf4592f37de35187 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3bf933b41ff74ce6bf4592f37de35187. Latest consensus state: current_term: 1 leader_uuid: "3bf933b41ff74ce6bf4592f37de35187" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bf933b41ff74ce6bf4592f37de35187" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41205 } } }
I20260811 17:23:29.957765 4742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bf933b41ff74ce6bf4592f37de35187 [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:29.957909 4753 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:23:29.961582 4753 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:23:29.968189 4753 catalog_manager.cc:1347] Generated new cluster ID: 41dbedd158c547788705b3cf3a9eff3e
I20260811 17:23:29.968286 4753 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:23:29.986718 4753 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:23:29.987762 4753 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:23:29.994432 4753 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3bf933b41ff74ce6bf4592f37de35187: Generated new TSK 0
I20260811 17:23:29.995273 4753 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:23:30.004863 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:0
--local_ip_for_outbound_sockets=127.9.157.129
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:41205
--builtin_ntp_servers=127.9.157.148:46095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 17:23:30.155998 4777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:30.156298 4777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:30.156340 4777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:30.160804 4777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:30.160890 4777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:30.160957 4777 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:23:30.166097 4777 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:46095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:41205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:30.167337 4777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:30.168865 4777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:30.179361 4792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:30.179358 4785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:30.179461 4786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:30.179580 4777 server_base.cc:1034] running on GCE node
I20260811 17:23:30.180181 4777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:30.180851 4777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:30.182050 4777 hybrid_clock.cc:648] HybridClock initialized: now 1786469010182031 us; error 29 us; skew 500 ppm
I20260811 17:23:30.184206 4777 webserver.cc:460] Webserver started at http://127.9.157.129:36081/ using document root <none> and password file <none>
I20260811 17:23:30.184839 4777 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:30.184906 4777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:30.185156 4777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:30.187446 4777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-0/data/instance:
uuid: "9f1ee6cd3b00478baa19f239b663347a"
format_stamp: "Formatted at 2026-08-11 17:23:30 on dist-test-slave-mjjr"
server_key: "7a558b8e046ac179475d15576459e438"
server_key_iv: "41491bf37861b9f51874531ceaa2319c"
server_key_version: "encryptionkey@0"
I20260811 17:23:30.188105 4777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance:
uuid: "9f1ee6cd3b00478baa19f239b663347a"
format_stamp: "Formatted at 2026-08-11 17:23:30 on dist-test-slave-mjjr"
server_key: "7a558b8e046ac179475d15576459e438"
server_key_iv: "41491bf37861b9f51874531ceaa2319c"
server_key_version: "encryptionkey@0"
I20260811 17:23:30.192265 4777 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:23:30.194911 4804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:30.196264 4777 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:23:30.196412 4777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "9f1ee6cd3b00478baa19f239b663347a"
format_stamp: "Formatted at 2026-08-11 17:23:30 on dist-test-slave-mjjr"
server_key: "7a558b8e046ac179475d15576459e438"
server_key_iv: "41491bf37861b9f51874531ceaa2319c"
server_key_version: "encryptionkey@0"
I20260811 17:23:30.196519 4777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:30.214236 4777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:30.215107 4777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:30.215288 4777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:30.215994 4777 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:23:30.217005 4777 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:23:30.217056 4777 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:30.217097 4777 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:23:30.217119 4777 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:30.226490 4777 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:39195
I20260811 17:23:30.226532 4977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:39195 every 8 connection(s)
I20260811 17:23:30.227494 4777 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:23:30.235244 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 4777
I20260811 17:23:30.235410 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance
I20260811 17:23:30.235759 9846 external_mini_cluster.cc:1428] Setting key 507fa1a42e40eb536d773f7d4e73ce12
I20260811 17:23:30.238188 4736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:30.229542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:45379 (local address 127.9.157.190:41205)
0811 17:23:30.229692 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:30.229696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:30.230853 (+ 1157us) server_negotiation.cc:407] Connection header received
0811 17:23:30.232312 (+ 1459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:30.232315 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:30.232373 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:30.232455 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:30.233364 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.233900 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.235172 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.235523 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.235641 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:30.237638 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:30.237642 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:30.237645 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:30.237663 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:30.237685 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:30.237694 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:30.237781 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:30.237967 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 17:23:30.238039 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 17:23:30.238189 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:0
--local_ip_for_outbound_sockets=127.9.157.130
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:41205
--builtin_ntp_servers=127.9.157.148:46095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 17:23:30.239293 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:30.229992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:41205 (local address 127.9.157.129:45379)
0811 17:23:30.230695 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:30.230739 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:30.232103 (+ 1364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:30.232622 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:30.232629 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:30.232713 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:23:30.233223 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:30.233237 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.234051 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.234054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:30.235004 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:30.235015 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.235733 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.235740 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:30.235983 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:30.237276 (+ 1293us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:23:30.237278 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:23:30.237345 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:23:30.237447 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:23:30.237786 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:23:30.237799 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:23:30.237806 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 17:23:30.237967 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 17:23:30.238313 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":123,"threads_started":1}
I20260811 17:23:30.240638 4978 heartbeater.cc:347] Connected to a master server at 127.9.157.190:41205
I20260811 17:23:30.240895 4978 heartbeater.cc:464] Registering TS with master...
I20260811 17:23:30.241467 4978 heartbeater.cc:511] Master 127.9.157.190:41205 requested a full tablet report, sending...
I20260811 17:23:30.243033 4675 ts_manager.cc:194] Registered new tserver with Master: 9f1ee6cd3b00478baa19f239b663347a (127.9.157.129:39195)
I20260811 17:23:30.244293 4675 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.157.129:45379
W20260811 17:23:30.360394 4981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:30.360749 4981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:30.360836 4981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:30.365109 4981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:30.365216 4981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:30.365315 4981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:23:30.370509 4981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:46095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:41205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:30.371866 4981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:30.373383 4981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:30.383997 4995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:30.383997 4989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:30.383997 4990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:30.384053 4981 server_base.cc:1034] running on GCE node
I20260811 17:23:30.384822 4981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:30.385623 4981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:30.386817 4981 hybrid_clock.cc:648] HybridClock initialized: now 1786469010386798 us; error 44 us; skew 500 ppm
I20260811 17:23:30.389078 4981 webserver.cc:460] Webserver started at http://127.9.157.130:36213/ using document root <none> and password file <none>
I20260811 17:23:30.389768 4981 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:30.389874 4981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:30.390130 4981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:30.392498 4981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-1/data/instance:
uuid: "7989343c868045d9be9663ac8f5f0766"
format_stamp: "Formatted at 2026-08-11 17:23:30 on dist-test-slave-mjjr"
server_key: "f46a1932dbba9e91ad8dd6a9b6f8fc6b"
server_key_iv: "1f08e3946d28c24eab9c4734577f3ddb"
server_key_version: "encryptionkey@0"
I20260811 17:23:30.393121 4981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance:
uuid: "7989343c868045d9be9663ac8f5f0766"
format_stamp: "Formatted at 2026-08-11 17:23:30 on dist-test-slave-mjjr"
server_key: "f46a1932dbba9e91ad8dd6a9b6f8fc6b"
server_key_iv: "1f08e3946d28c24eab9c4734577f3ddb"
server_key_version: "encryptionkey@0"
I20260811 17:23:30.397177 4981 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:23:30.399923 5007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:30.401342 4981 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 17:23:30.401453 4981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "7989343c868045d9be9663ac8f5f0766"
format_stamp: "Formatted at 2026-08-11 17:23:30 on dist-test-slave-mjjr"
server_key: "f46a1932dbba9e91ad8dd6a9b6f8fc6b"
server_key_iv: "1f08e3946d28c24eab9c4734577f3ddb"
server_key_version: "encryptionkey@0"
I20260811 17:23:30.401549 4981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:30.419622 4981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:30.420488 4981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:30.420655 4981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:30.421262 4981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:23:30.422253 4981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:23:30.422303 4981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:30.422348 4981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:23:30.422363 4981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:30.432107 4981 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:34889
I20260811 17:23:30.432137 5186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:34889 every 8 connection(s)
I20260811 17:23:30.433121 4981 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:23:30.436975 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 4981
I20260811 17:23:30.437117 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance
I20260811 17:23:30.437415 9846 external_mini_cluster.cc:1428] Setting key de403318f190b4bb87a7fc839cd2d641
I20260811 17:23:30.439851 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:0
--local_ip_for_outbound_sockets=127.9.157.131
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:41205
--builtin_ntp_servers=127.9.157.148:46095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 17:23:30.444387 4736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:30.435402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:45801 (local address 127.9.157.190:41205)
0811 17:23:30.435597 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:30.435601 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:30.436884 (+ 1283us) server_negotiation.cc:407] Connection header received
0811 17:23:30.438607 (+ 1723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:30.438610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:30.438658 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:30.438733 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:30.439988 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.440534 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.441814 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.442120 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.442229 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:30.443793 (+ 1564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:30.443797 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:30.443799 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:30.443815 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:30.443838 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:30.443847 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:30.443920 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:30.444183 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 17:23:30.444251 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 17:23:30.445757 5190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:30.435926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:41205 (local address 127.9.157.130:45801)
0811 17:23:30.436690 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:30.436735 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:30.438350 (+ 1615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:30.439077 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:30.439090 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:30.439210 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:23:30.439827 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:30.439843 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.440683 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.440691 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:30.441617 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:30.441651 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.442324 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.442331 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:30.442581 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:30.443369 (+ 788us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:23:30.443373 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:23:30.443436 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:23:30.443539 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:23:30.443948 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:23:30.443962 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:23:30.443972 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 17:23:30.444122 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 17:23:30.444494 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":133,"threads_started":1}
I20260811 17:23:30.447199 5188 heartbeater.cc:347] Connected to a master server at 127.9.157.190:41205
I20260811 17:23:30.447510 5188 heartbeater.cc:464] Registering TS with master...
I20260811 17:23:30.448177 5188 heartbeater.cc:511] Master 127.9.157.190:41205 requested a full tablet report, sending...
I20260811 17:23:30.449345 4675 ts_manager.cc:194] Registered new tserver with Master: 7989343c868045d9be9663ac8f5f0766 (127.9.157.130:34889)
I20260811 17:23:30.449961 4675 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.157.130:45801
W20260811 17:23:30.563776 5192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:30.564098 5192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:30.564139 5192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:30.568563 5192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:30.568640 5192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:30.568702 5192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:23:30.573817 5192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:46095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:41205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:30.575129 5192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:30.576673 5192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:30.587055 5209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:30.587055 5204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:30.587073 5205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:30.587548 5192 server_base.cc:1034] running on GCE node
I20260811 17:23:30.588035 5192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:30.588795 5192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:30.589996 5192 hybrid_clock.cc:648] HybridClock initialized: now 1786469010589960 us; error 54 us; skew 500 ppm
I20260811 17:23:30.592217 5192 webserver.cc:460] Webserver started at http://127.9.157.131:44901/ using document root <none> and password file <none>
I20260811 17:23:30.592866 5192 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:30.592938 5192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:30.593173 5192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:30.595441 5192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-2/data/instance:
uuid: "37f31607a2944606ae39ab84f31d4962"
format_stamp: "Formatted at 2026-08-11 17:23:30 on dist-test-slave-mjjr"
server_key: "d3dfd83ed9104ef4817c5ebbc0a3bf26"
server_key_iv: "e891cfa0716bbf0642f9160bccf32a1d"
server_key_version: "encryptionkey@0"
I20260811 17:23:30.596141 5192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance:
uuid: "37f31607a2944606ae39ab84f31d4962"
format_stamp: "Formatted at 2026-08-11 17:23:30 on dist-test-slave-mjjr"
server_key: "d3dfd83ed9104ef4817c5ebbc0a3bf26"
server_key_iv: "e891cfa0716bbf0642f9160bccf32a1d"
server_key_version: "encryptionkey@0"
I20260811 17:23:30.600786 5192 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:23:30.603811 5221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:30.605448 5192 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:23:30.605589 5192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "37f31607a2944606ae39ab84f31d4962"
format_stamp: "Formatted at 2026-08-11 17:23:30 on dist-test-slave-mjjr"
server_key: "d3dfd83ed9104ef4817c5ebbc0a3bf26"
server_key_iv: "e891cfa0716bbf0642f9160bccf32a1d"
server_key_version: "encryptionkey@0"
I20260811 17:23:30.605785 5192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:30.634967 5192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:30.635767 5192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:30.635936 5192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:30.636560 5192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:23:30.637565 5192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:23:30.637616 5192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:30.637658 5192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:23:30.637674 5192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:30.647517 5192 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:43101
I20260811 17:23:30.647557 5403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:43101 every 8 connection(s)
I20260811 17:23:30.648535 5192 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:23:30.649410 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 5192
I20260811 17:23:30.649550 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance
I20260811 17:23:30.649839 9846 external_mini_cluster.cc:1428] Setting key f9f5f214f33a64deab567491ea89950c
I20260811 17:23:30.658660 4736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:30.650573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:54835 (local address 127.9.157.190:41205)
0811 17:23:30.650730 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:30.650734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:30.652172 (+ 1438us) server_negotiation.cc:407] Connection header received
0811 17:23:30.653715 (+ 1543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:30.653718 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:30.653767 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:30.653846 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:30.654773 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.655305 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.656573 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.656915 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.656996 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:30.658221 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:30.658224 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:30.658226 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:30.658240 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:30.658258 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:30.658266 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:30.658329 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:30.658479 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 17:23:30.658535 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 17:23:30.659588 5406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:30.651152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:41205 (local address 127.9.157.131:54835)
0811 17:23:30.652020 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:30.652076 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:30.653510 (+ 1434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:30.654022 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:30.654029 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:30.654116 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:23:30.654650 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:30.654664 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.655418 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.655420 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:30.656450 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:30.656462 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.657068 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.657071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:30.657269 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:30.657953 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:23:30.657956 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:23:30.658000 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:23:30.658074 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:23:30.658351 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:23:30.658359 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:23:30.658364 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 17:23:30.658467 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 17:23:30.658714 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":165,"threads_started":1}
I20260811 17:23:30.660852 5404 heartbeater.cc:347] Connected to a master server at 127.9.157.190:41205
I20260811 17:23:30.661144 5404 heartbeater.cc:464] Registering TS with master...
I20260811 17:23:30.661800 5404 heartbeater.cc:511] Master 127.9.157.190:41205 requested a full tablet report, sending...
I20260811 17:23:30.663076 4675 ts_manager.cc:194] Registered new tserver with Master: 37f31607a2944606ae39ab84f31d4962 (127.9.157.131:43101)
I20260811 17:23:30.663873 4675 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.157.131:54835
I20260811 17:23:30.664361 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:23:30.670408 4736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:30.666579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57526 (local address 127.9.157.190:41205)
0811 17:23:30.666704 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:30.666707 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:30.666905 (+ 198us) server_negotiation.cc:407] Connection header received
0811 17:23:30.667018 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:30.667020 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:30.667061 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:30.667131 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:30.667418 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.668014 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.668864 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.669062 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.669136 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:30.669974 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:30.669977 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:30.669979 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:30.669992 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:30.670011 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:30.670018 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:30.670088 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:30.670233 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 17:23:30.670287 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 17:23:30.673872 4675 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57526:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:23:30.677230 4675 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:23:30.692198 5421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:30.686374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35318 (local address 127.9.157.131:43101)
0811 17:23:30.687272 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:30.687289 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:30.687319 (+ 30us) server_negotiation.cc:407] Connection header received
0811 17:23:30.687410 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:30.687415 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:30.687639 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:30.688222 (+ 583us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:30.688397 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.689379 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.690822 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.691367 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.691490 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:30.691734 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:30.691914 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 17:23:30.692022 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":768,"thread_start_us":125,"threads_started":1}
I20260811 17:23:30.692193 5419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:30.686252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:43101 (local address 127.0.0.1:35318)
0811 17:23:30.686756 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:30.686809 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:30.686941 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:30.687852 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:30.687859 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:30.687885 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:30.688056 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:30.688066 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.689493 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.689498 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:30.690651 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:30.690663 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.691502 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.691505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:30.691634 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:30.691721 (+ 87us) client_negotiation.cc:769] Sending connection context
0811 17:23:30.691875 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 17:23:30.692014 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":1,"thread_start_us":124,"threads_started":1}
I20260811 17:23:30.692592 5418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:30.686322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:39195 (local address 127.0.0.1:38548)
0811 17:23:30.687221 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:30.687237 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:30.687332 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:30.688450 (+ 1118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:30.688453 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:30.688472 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:30.688582 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:30.688592 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.690095 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.690099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:30.691117 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:30.691128 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.692214 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.692218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:30.692303 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:30.692330 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 17:23:30.692379 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 17:23:30.692453 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":818,"mutex_wait_us":30,"thread_start_us":140,"threads_started":1}
I20260811 17:23:30.693347 5422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:30.687167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38548 (local address 127.9.157.129:39195)
0811 17:23:30.687989 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:30.688003 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:30.688030 (+ 27us) server_negotiation.cc:407] Connection header received
0811 17:23:30.688112 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:30.688119 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:30.688303 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:30.688468 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:30.688726 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.689946 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.691252 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.692076 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.692795 (+ 719us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:30.693002 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:30.693056 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 17:23:30.693165 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":165,"threads_started":1}
I20260811 17:23:30.694324 5424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:30.687724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:34889 (local address 127.0.0.1:52362)
0811 17:23:30.688082 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:30.688092 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:30.688208 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:30.688691 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:30.688695 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:30.688715 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:30.688805 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:30.688811 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.690136 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.690139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:30.692291 (+ 2152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:30.692300 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.693843 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.693847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:30.693935 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:30.693959 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 17:23:30.694037 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 17:23:30.694106 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":74,"thread_start_us":135,"threads_started":1}
I20260811 17:23:30.694514 5423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:30.687465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52362 (local address 127.9.157.130:34889)
0811 17:23:30.687964 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:30.687979 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:30.688191 (+ 212us) server_negotiation.cc:407] Connection header received
0811 17:23:30.688347 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:30.688357 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:30.688537 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:30.688701 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:30.688926 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.690020 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.692441 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.693204 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.693329 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:30.693528 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:30.694204 (+ 676us) server_negotiation.cc:299] Negotiation successful
0811 17:23:30.694332 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":132,"threads_started":1}
I20260811 17:23:30.696040 5297 tablet_service.cc:1467] Processing CreateTablet for tablet 9011059cf26c40c38c3ae65704815c81 (DEFAULT_TABLE table=test-table [id=0caae1687f6b4cd091391e111846e992]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:23:30.697297 5297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9011059cf26c40c38c3ae65704815c81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:30.697467 5080 tablet_service.cc:1467] Processing CreateTablet for tablet 9011059cf26c40c38c3ae65704815c81 (DEFAULT_TABLE table=test-table [id=0caae1687f6b4cd091391e111846e992]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:23:30.698544 5080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9011059cf26c40c38c3ae65704815c81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:30.698386 4881 tablet_service.cc:1467] Processing CreateTablet for tablet 9011059cf26c40c38c3ae65704815c81 (DEFAULT_TABLE table=test-table [id=0caae1687f6b4cd091391e111846e992]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:23:30.699489 4881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9011059cf26c40c38c3ae65704815c81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:30.705041 5426 tablet_bootstrap.cc:492] T 9011059cf26c40c38c3ae65704815c81 P 7989343c868045d9be9663ac8f5f0766: Bootstrap starting.
I20260811 17:23:30.706453 5427 tablet_bootstrap.cc:492] T 9011059cf26c40c38c3ae65704815c81 P 37f31607a2944606ae39ab84f31d4962: Bootstrap starting.
I20260811 17:23:30.707100 5428 tablet_bootstrap.cc:492] T 9011059cf26c40c38c3ae65704815c81 P 9f1ee6cd3b00478baa19f239b663347a: Bootstrap starting.
I20260811 17:23:30.707683 5426 tablet_bootstrap.cc:654] T 9011059cf26c40c38c3ae65704815c81 P 7989343c868045d9be9663ac8f5f0766: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:30.708748 5427 tablet_bootstrap.cc:654] T 9011059cf26c40c38c3ae65704815c81 P 37f31607a2944606ae39ab84f31d4962: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:30.708757 5426 log.cc:824] T 9011059cf26c40c38c3ae65704815c81 P 7989343c868045d9be9663ac8f5f0766: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:30.709545 5428 tablet_bootstrap.cc:654] T 9011059cf26c40c38c3ae65704815c81 P 9f1ee6cd3b00478baa19f239b663347a: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:30.709805 5427 log.cc:824] T 9011059cf26c40c38c3ae65704815c81 P 37f31607a2944606ae39ab84f31d4962: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:30.710706 5428 log.cc:824] T 9011059cf26c40c38c3ae65704815c81 P 9f1ee6cd3b00478baa19f239b663347a: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:30.711501 5426 tablet_bootstrap.cc:492] T 9011059cf26c40c38c3ae65704815c81 P 7989343c868045d9be9663ac8f5f0766: No bootstrap required, opened a new log
I20260811 17:23:30.711799 5426 ts_tablet_manager.cc:1397] T 9011059cf26c40c38c3ae65704815c81 P 7989343c868045d9be9663ac8f5f0766: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 17:23:30.712036 5427 tablet_bootstrap.cc:492] T 9011059cf26c40c38c3ae65704815c81 P 37f31607a2944606ae39ab84f31d4962: No bootstrap required, opened a new log
I20260811 17:23:30.712252 5427 ts_tablet_manager.cc:1397] T 9011059cf26c40c38c3ae65704815c81 P 37f31607a2944606ae39ab84f31d4962: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 17:23:30.713037 5428 tablet_bootstrap.cc:492] T 9011059cf26c40c38c3ae65704815c81 P 9f1ee6cd3b00478baa19f239b663347a: No bootstrap required, opened a new log
I20260811 17:23:30.713251 5428 ts_tablet_manager.cc:1397] T 9011059cf26c40c38c3ae65704815c81 P 9f1ee6cd3b00478baa19f239b663347a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 17:23:30.715778 5427 raft_consensus.cc:348] T 9011059cf26c40c38c3ae65704815c81 P 37f31607a2944606ae39ab84f31d4962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37f31607a2944606ae39ab84f31d4962" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 43101 } } peers { permanent_uuid: "9f1ee6cd3b00478baa19f239b663347a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39195 } } peers { permanent_uuid: "7989343c868045d9be9663ac8f5f0766" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 34889 } }
I20260811 17:23:30.716228 5427 raft_consensus.cc:374] T 9011059cf26c40c38c3ae65704815c81 P 37f31607a2944606ae39ab84f31d4962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:30.716377 5427 raft_consensus.cc:729] T 9011059cf26c40c38c3ae65704815c81 P 37f31607a2944606ae39ab84f31d4962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37f31607a2944606ae39ab84f31d4962, State: Initialized, Role: FOLLOWER
I20260811 17:23:30.716675 5426 raft_consensus.cc:348] T 9011059cf26c40c38c3ae65704815c81 P 7989343c868045d9be9663ac8f5f0766 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37f31607a2944606ae39ab84f31d4962" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 43101 } } peers { permanent_uuid: "9f1ee6cd3b00478baa19f239b663347a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39195 } } peers { permanent_uuid: "7989343c868045d9be9663ac8f5f0766" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 34889 } }
I20260811 17:23:30.716980 5426 raft_consensus.cc:374] T 9011059cf26c40c38c3ae65704815c81 P 7989343c868045d9be9663ac8f5f0766 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:30.716964 5427 consensus_queue.cc:260] T 9011059cf26c40c38c3ae65704815c81 P 37f31607a2944606ae39ab84f31d4962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37f31607a2944606ae39ab84f31d4962" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 43101 } } peers { permanent_uuid: "9f1ee6cd3b00478baa19f239b663347a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39195 } } peers { permanent_uuid: "7989343c868045d9be9663ac8f5f0766" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 34889 } }
I20260811 17:23:30.717092 5426 raft_consensus.cc:729] T 9011059cf26c40c38c3ae65704815c81 P 7989343c868045d9be9663ac8f5f0766 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7989343c868045d9be9663ac8f5f0766, State: Initialized, Role: FOLLOWER
I20260811 17:23:30.717823 5404 heartbeater.cc:503] Master 127.9.157.190:41205 was elected leader, sending a full tablet report...
I20260811 17:23:30.717818 5426 consensus_queue.cc:260] T 9011059cf26c40c38c3ae65704815c81 P 7989343c868045d9be9663ac8f5f0766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37f31607a2944606ae39ab84f31d4962" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 43101 } } peers { permanent_uuid: "9f1ee6cd3b00478baa19f239b663347a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39195 } } peers { permanent_uuid: "7989343c868045d9be9663ac8f5f0766" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 34889 } }
I20260811 17:23:30.718118 5427 ts_tablet_manager.cc:1428] T 9011059cf26c40c38c3ae65704815c81 P 37f31607a2944606ae39ab84f31d4962: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 17:23:30.718321 5428 raft_consensus.cc:348] T 9011059cf26c40c38c3ae65704815c81 P 9f1ee6cd3b00478baa19f239b663347a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37f31607a2944606ae39ab84f31d4962" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 43101 } } peers { permanent_uuid: "9f1ee6cd3b00478baa19f239b663347a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39195 } } peers { permanent_uuid: "7989343c868045d9be9663ac8f5f0766" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 34889 } }
I20260811 17:23:30.718532 5188 heartbeater.cc:503] Master 127.9.157.190:41205 was elected leader, sending a full tablet report...
I20260811 17:23:30.718663 5428 raft_consensus.cc:374] T 9011059cf26c40c38c3ae65704815c81 P 9f1ee6cd3b00478baa19f239b663347a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:30.718768 5428 raft_consensus.cc:729] T 9011059cf26c40c38c3ae65704815c81 P 9f1ee6cd3b00478baa19f239b663347a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1ee6cd3b00478baa19f239b663347a, State: Initialized, Role: FOLLOWER
I20260811 17:23:30.718855 5426 ts_tablet_manager.cc:1428] T 9011059cf26c40c38c3ae65704815c81 P 7989343c868045d9be9663ac8f5f0766: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 17:23:30.719336 5428 consensus_queue.cc:260] T 9011059cf26c40c38c3ae65704815c81 P 9f1ee6cd3b00478baa19f239b663347a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37f31607a2944606ae39ab84f31d4962" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 43101 } } peers { permanent_uuid: "9f1ee6cd3b00478baa19f239b663347a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39195 } } peers { permanent_uuid: "7989343c868045d9be9663ac8f5f0766" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 34889 } }
I20260811 17:23:30.720063 4978 heartbeater.cc:503] Master 127.9.157.190:41205 was elected leader, sending a full tablet report...
I20260811 17:23:30.721606 5428 ts_tablet_manager.cc:1428] T 9011059cf26c40c38c3ae65704815c81 P 9f1ee6cd3b00478baa19f239b663347a: Time spent starting tablet: real 0.008s user 0.003s sys 0.005s
W20260811 17:23:30.728829 4979 tablet.cc:2369] T 9011059cf26c40c38c3ae65704815c81 P 9f1ee6cd3b00478baa19f239b663347a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:23:30.735719 5441 raft_consensus.cc:482] T 9011059cf26c40c38c3ae65704815c81 P 9f1ee6cd3b00478baa19f239b663347a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:23:30.735970 5441 raft_consensus.cc:504] T 9011059cf26c40c38c3ae65704815c81 P 9f1ee6cd3b00478baa19f239b663347a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37f31607a2944606ae39ab84f31d4962" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 43101 } } peers { permanent_uuid: "9f1ee6cd3b00478baa19f239b663347a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39195 } } peers { permanent_uuid: "7989343c868045d9be9663ac8f5f0766" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 34889 } }
I20260811 17:23:30.737112 5441 leader_election.cc:288] T 9011059cf26c40c38c3ae65704815c81 P 9f1ee6cd3b00478baa19f239b663347a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 37f31607a2944606ae39ab84f31d4962 (127.9.157.131:43101), 7989343c868045d9be9663ac8f5f0766 (127.9.157.130:34889)
I20260811 17:23:30.740861 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:30.737407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:43101 (local address 127.9.157.129:44595)
0811 17:23:30.737563 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:30.737581 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:30.737689 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:30.737978 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:30.737981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:30.738001 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:30.738122 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:30.738128 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.738872 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.738876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:30.739739 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:30.739749 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.740410 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.740413 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:30.740520 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:30.740547 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 17:23:30.740634 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 17:23:30.740711 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 17:23:30.740861 5421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:30.737497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:44595 (local address 127.9.157.131:43101)
0811 17:23:30.737652 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:30.737657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:30.737673 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:23:30.737811 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:30.737814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:30.737868 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:30.737952 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:30.738230 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.738751 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.739870 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.740273 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.740410 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:30.740464 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:30.740654 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 17:23:30.740730 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 17:23:30.741614 5442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:30.737744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:34889 (local address 127.9.157.129:59941)
0811 17:23:30.738208 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:30.738220 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:30.738342 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:30.738601 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:30.738604 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:30.738621 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:30.738757 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:30.738763 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.739495 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.739499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:30.740379 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:30.740390 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.741193 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.741199 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:30.741299 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:30.741319 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 17:23:30.741406 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 17:23:30.741475 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":156,"threads_started":1}
I20260811 17:23:30.741626 5423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:30.737894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:59941 (local address 127.9.157.130:34889)
0811 17:23:30.738053 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:30.738060 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:30.738311 (+ 251us) server_negotiation.cc:407] Connection header received
0811 17:23:30.738454 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:30.738457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:30.738507 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:30.738589 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:30.738905 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.739380 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.740514 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.740915 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.741075 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:30.741134 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:30.741433 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 17:23:30.741518 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 17:23:30.741652 5329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9011059cf26c40c38c3ae65704815c81" candidate_uuid: "9f1ee6cd3b00478baa19f239b663347a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37f31607a2944606ae39ab84f31d4962" is_pre_election: true
I20260811 17:23:30.742084 5329 raft_consensus.cc:2454] T 9011059cf26c40c38c3ae65704815c81 P 37f31607a2944606ae39ab84f31d4962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1ee6cd3b00478baa19f239b663347a in term 0.
I20260811 17:23:30.742197 5113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9011059cf26c40c38c3ae65704815c81" candidate_uuid: "9f1ee6cd3b00478baa19f239b663347a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7989343c868045d9be9663ac8f5f0766" is_pre_election: true
I20260811 17:23:30.742532 5113 raft_consensus.cc:2454] T 9011059cf26c40c38c3ae65704815c81 P 7989343c868045d9be9663ac8f5f0766 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1ee6cd3b00478baa19f239b663347a in term 0.
I20260811 17:23:30.742678 4809 leader_election.cc:302] T 9011059cf26c40c38c3ae65704815c81 P 9f1ee6cd3b00478baa19f239b663347a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37f31607a2944606ae39ab84f31d4962, 9f1ee6cd3b00478baa19f239b663347a; no voters:
I20260811 17:23:30.742993 5441 raft_consensus.cc:2781] T 9011059cf26c40c38c3ae65704815c81 P 9f1ee6cd3b00478baa19f239b663347a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:23:30.743079 5441 raft_consensus.cc:482] T 9011059cf26c40c38c3ae65704815c81 P 9f1ee6cd3b00478baa19f239b663347a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:23:30.743110 5441 raft_consensus.cc:3037] T 9011059cf26c40c38c3ae65704815c81 P 9f1ee6cd3b00478baa19f239b663347a [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:30.744345 5441 raft_consensus.cc:504] T 9011059cf26c40c38c3ae65704815c81 P 9f1ee6cd3b00478baa19f239b663347a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37f31607a2944606ae39ab84f31d4962" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 43101 } } peers { permanent_uuid: "9f1ee6cd3b00478baa19f239b663347a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39195 } } peers { permanent_uuid: "7989343c868045d9be9663ac8f5f0766" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 34889 } }
I20260811 17:23:30.744711 5441 leader_election.cc:288] T 9011059cf26c40c38c3ae65704815c81 P 9f1ee6cd3b00478baa19f239b663347a [CANDIDATE]: Term 1 election: Requested vote from peers 37f31607a2944606ae39ab84f31d4962 (127.9.157.131:43101), 7989343c868045d9be9663ac8f5f0766 (127.9.157.130:34889)
I20260811 17:23:30.745323 5113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9011059cf26c40c38c3ae65704815c81" candidate_uuid: "9f1ee6cd3b00478baa19f239b663347a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7989343c868045d9be9663ac8f5f0766"
I20260811 17:23:30.745323 5329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9011059cf26c40c38c3ae65704815c81" candidate_uuid: "9f1ee6cd3b00478baa19f239b663347a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37f31607a2944606ae39ab84f31d4962"
I20260811 17:23:30.745476 5329 raft_consensus.cc:3037] T 9011059cf26c40c38c3ae65704815c81 P 37f31607a2944606ae39ab84f31d4962 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:30.745476 5113 raft_consensus.cc:3037] T 9011059cf26c40c38c3ae65704815c81 P 7989343c868045d9be9663ac8f5f0766 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:30.746897 5113 raft_consensus.cc:2454] T 9011059cf26c40c38c3ae65704815c81 P 7989343c868045d9be9663ac8f5f0766 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1ee6cd3b00478baa19f239b663347a in term 1.
I20260811 17:23:30.746894 5329 raft_consensus.cc:2454] T 9011059cf26c40c38c3ae65704815c81 P 37f31607a2944606ae39ab84f31d4962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1ee6cd3b00478baa19f239b663347a in term 1.
I20260811 17:23:30.747238 4809 leader_election.cc:302] T 9011059cf26c40c38c3ae65704815c81 P 9f1ee6cd3b00478baa19f239b663347a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37f31607a2944606ae39ab84f31d4962, 9f1ee6cd3b00478baa19f239b663347a; no voters:
I20260811 17:23:30.747457 5441 raft_consensus.cc:2781] T 9011059cf26c40c38c3ae65704815c81 P 9f1ee6cd3b00478baa19f239b663347a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:23:30.747748 5441 raft_consensus.cc:686] T 9011059cf26c40c38c3ae65704815c81 P 9f1ee6cd3b00478baa19f239b663347a [term 1 LEADER]: Becoming Leader. State: Replica: 9f1ee6cd3b00478baa19f239b663347a, State: Running, Role: LEADER
I20260811 17:23:30.748145 5441 consensus_queue.cc:237] T 9011059cf26c40c38c3ae65704815c81 P 9f1ee6cd3b00478baa19f239b663347a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37f31607a2944606ae39ab84f31d4962" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 43101 } } peers { permanent_uuid: "9f1ee6cd3b00478baa19f239b663347a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39195 } } peers { permanent_uuid: "7989343c868045d9be9663ac8f5f0766" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 34889 } }
I20260811 17:23:30.751477 4675 catalog_manager.cc:5495] T 9011059cf26c40c38c3ae65704815c81 P 9f1ee6cd3b00478baa19f239b663347a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f1ee6cd3b00478baa19f239b663347a (127.9.157.129). New cstate: current_term: 1 leader_uuid: "9f1ee6cd3b00478baa19f239b663347a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37f31607a2944606ae39ab84f31d4962" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 43101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1ee6cd3b00478baa19f239b663347a" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 39195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7989343c868045d9be9663ac8f5f0766" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 34889 } health_report { overall_health: UNKNOWN } } }
I20260811 17:23:30.772908 5422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:30.769448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38552 (local address 127.9.157.129:39195)
0811 17:23:30.769578 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:30.769582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:30.769594 (+ 12us) server_negotiation.cc:407] Connection header received
0811 17:23:30.769675 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:30.769677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:30.769738 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:30.769839 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:23:30.770051 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.770575 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.771432 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:30.771670 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:30.771751 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:30.772121 (+ 370us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:23:30.772561 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:23:30.772651 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:30.772734 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 17:23:30.772786 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 17:23:30.779510 5329 raft_consensus.cc:1261] T 9011059cf26c40c38c3ae65704815c81 P 37f31607a2944606ae39ab84f31d4962 [term 1 FOLLOWER]: Refusing update from remote peer 9f1ee6cd3b00478baa19f239b663347a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:23:30.779522 5113 raft_consensus.cc:1261] T 9011059cf26c40c38c3ae65704815c81 P 7989343c868045d9be9663ac8f5f0766 [term 1 FOLLOWER]: Refusing update from remote peer 9f1ee6cd3b00478baa19f239b663347a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:23:30.780335 5443 consensus_queue.cc:1035] T 9011059cf26c40c38c3ae65704815c81 P 9f1ee6cd3b00478baa19f239b663347a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7989343c868045d9be9663ac8f5f0766" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:23:30.780593 5441 consensus_queue.cc:1035] T 9011059cf26c40c38c3ae65704815c81 P 9f1ee6cd3b00478baa19f239b663347a [LEADER]: Connected to new peer: Peer: permanent_uuid: "37f31607a2944606ae39ab84f31d4962" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 43101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:23:30.790540 5450 mvcc.cc:205] Tried to move back new op lower bound from 7317377068145074176 to 7317377068027650048. Current Snapshot: MvccSnapshot[applied={T|T < 7317377068145074176}]
I20260811 17:23:30.798026 4675 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:23:30.798242 4675 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:23:30.801244 4675 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3bf933b41ff74ce6bf4592f37de35187: Sending DeleteTablet for 3 replicas of tablet 9011059cf26c40c38c3ae65704815c81
I20260811 17:23:30.802192 4881 tablet_service.cc:1514] Processing DeleteTablet for tablet 9011059cf26c40c38c3ae65704815c81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:23:30 UTC) from {username='slave'} at 127.0.0.1:38548
I20260811 17:23:30.802428 5297 tablet_service.cc:1514] Processing DeleteTablet for tablet 9011059cf26c40c38c3ae65704815c81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:23:30 UTC) from {username='slave'} at 127.0.0.1:35318
I20260811 17:23:30.803015 5462 tablet_replica.cc:330] T 9011059cf26c40c38c3ae65704815c81 P 37f31607a2944606ae39ab84f31d4962: stopping tablet replica
I20260811 17:23:30.803179 5080 tablet_service.cc:1514] Processing DeleteTablet for tablet 9011059cf26c40c38c3ae65704815c81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:23:30 UTC) from {username='slave'} at 127.0.0.1:52362
I20260811 17:23:30.803334 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 4777
I20260811 17:23:30.810554 5462 raft_consensus.cc:2229] T 9011059cf26c40c38c3ae65704815c81 P 37f31607a2944606ae39ab84f31d4962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:23:30.811004 5462 raft_consensus.cc:2258] T 9011059cf26c40c38c3ae65704815c81 P 37f31607a2944606ae39ab84f31d4962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:23:30.812098 5465 tablet_replica.cc:330] T 9011059cf26c40c38c3ae65704815c81 P 7989343c868045d9be9663ac8f5f0766: stopping tablet replica
I20260811 17:23:30.812521 5465 raft_consensus.cc:2229] T 9011059cf26c40c38c3ae65704815c81 P 7989343c868045d9be9663ac8f5f0766 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:23:30.812928 5465 raft_consensus.cc:2258] T 9011059cf26c40c38c3ae65704815c81 P 7989343c868045d9be9663ac8f5f0766 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:23:30.813547 5462 ts_tablet_manager.cc:1905] T 9011059cf26c40c38c3ae65704815c81 P 37f31607a2944606ae39ab84f31d4962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:23:30.814673 5465 ts_tablet_manager.cc:1905] T 9011059cf26c40c38c3ae65704815c81 P 7989343c868045d9be9663ac8f5f0766: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:23:30.816994 5462 ts_tablet_manager.cc:1918] T 9011059cf26c40c38c3ae65704815c81 P 37f31607a2944606ae39ab84f31d4962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:23:30.817145 5462 log.cc:1196] T 9011059cf26c40c38c3ae65704815c81 P 37f31607a2944606ae39ab84f31d4962: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-2/wal/wals/9011059cf26c40c38c3ae65704815c81
I20260811 17:23:30.817551 5462 ts_tablet_manager.cc:1939] T 9011059cf26c40c38c3ae65704815c81 P 37f31607a2944606ae39ab84f31d4962: Deleting consensus metadata
I20260811 17:23:30.818106 5465 ts_tablet_manager.cc:1918] T 9011059cf26c40c38c3ae65704815c81 P 7989343c868045d9be9663ac8f5f0766: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:23:30.818230 5465 log.cc:1196] T 9011059cf26c40c38c3ae65704815c81 P 7989343c868045d9be9663ac8f5f0766: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786468743878666-9846-0/minicluster-data/ts-1/wal/wals/9011059cf26c40c38c3ae65704815c81
I20260811 17:23:30.818586 5465 ts_tablet_manager.cc:1939] T 9011059cf26c40c38c3ae65704815c81 P 7989343c868045d9be9663ac8f5f0766: Deleting consensus metadata
I20260811 17:23:30.819386 4655 catalog_manager.cc:4841] TS 37f31607a2944606ae39ab84f31d4962 (127.9.157.131:43101): tablet 9011059cf26c40c38c3ae65704815c81 (table test-table [id=0caae1687f6b4cd091391e111846e992]) successfully deleted
I20260811 17:23:30.819957 4655 catalog_manager.cc:4841] TS 7989343c868045d9be9663ac8f5f0766 (127.9.157.130:34889): tablet 9011059cf26c40c38c3ae65704815c81 (table test-table [id=0caae1687f6b4cd091391e111846e992]) successfully deleted
W20260811 17:23:30.820333 4652 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.157.129:39195 (error 108)
I20260811 17:23:30.820637 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 4981
I20260811 17:23:30.824436 5424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:30.823965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:39195 (local address 127.0.0.1:38556)
0811 17:23:30.824221 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:30.824315 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.129:39195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
W20260811 17:23:30.824648 4652 catalog_manager.cc:4584] TS 9f1ee6cd3b00478baa19f239b663347a (127.9.157.129:39195): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9011059cf26c40c38c3ae65704815c81: Network error: Client connection negotiation failed: client connection to 127.9.157.129:39195: connect: Connection refused (error 111)
I20260811 17:23:30.836040 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 5192
I20260811 17:23:30.853027 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 4630
2026-08-11T17:23:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1171 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:23:30 dist-test-slave-mjjr krb5kdc[5471](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:23:30 dist-test-slave-mjjr krb5kdc[5471](info): set up 2 sockets
Aug 11 17:23:30 dist-test-slave-mjjr krb5kdc[5471](info): commencing operation
krb5kdc: starting...
W20260811 17:23:32.127830 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:23:32 dist-test-slave-mjjr krb5kdc[5471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:23:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:23:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:23:32.267524 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:41139
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:41139
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:23:32.388995 5492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:32.389382 5492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:32.389496 5492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:32.393968 5492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:23:32.394090 5492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:32.394132 5492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:32.394186 5492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:23:32.394235 5492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:23:32.400056 5492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:41139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:41139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:32.401365 5492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:32.402506 5492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:23:32.411957 5492 server_base.cc:1034] running on GCE node
W20260811 17:23:32.411861 5499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:32.411861 5504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:32.411864 5500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:32.412863 5492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:32.414112 5492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:32.415282 5492 hybrid_clock.cc:648] HybridClock initialized: now 1786469012415247 us; error 50 us; skew 500 ppm
Aug 11 17:23:32 dist-test-slave-mjjr krb5kdc[5471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469012, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:23:32.418391 5492 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:23:32.419402 5492 webserver.cc:460] Webserver started at http://127.9.157.190:36953/ using document root <none> and password file <none>
I20260811 17:23:32.420046 5492 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:32.420117 5492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:32.420297 5492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:32.422573 5492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "a62837baccfc4b83a60b873b369e27d6"
format_stamp: "Formatted at 2026-08-11 17:23:32 on dist-test-slave-mjjr"
server_key: "ae010faa22443177e1724f02144e799d"
server_key_iv: "5059184215fccea17eb1d0c43603e8f4"
server_key_version: "encryptionkey@0"
I20260811 17:23:32.423135 5492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "a62837baccfc4b83a60b873b369e27d6"
format_stamp: "Formatted at 2026-08-11 17:23:32 on dist-test-slave-mjjr"
server_key: "ae010faa22443177e1724f02144e799d"
server_key_iv: "5059184215fccea17eb1d0c43603e8f4"
server_key_version: "encryptionkey@0"
I20260811 17:23:32.427208 5492 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:23:32.429911 5512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:32.431313 5492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:23:32.431483 5492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "a62837baccfc4b83a60b873b369e27d6"
format_stamp: "Formatted at 2026-08-11 17:23:32 on dist-test-slave-mjjr"
server_key: "ae010faa22443177e1724f02144e799d"
server_key_iv: "5059184215fccea17eb1d0c43603e8f4"
server_key_version: "encryptionkey@0"
I20260811 17:23:32.431614 5492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:32.446545 5492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:32.450145 5492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:32.450330 5492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:32.458151 5492 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:41139
I20260811 17:23:32.458168 5602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:41139 every 8 connection(s)
I20260811 17:23:32.459182 5492 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:23:32.462294 5603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:32.465935 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 5492
I20260811 17:23:32.466070 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:23:32.466394 9846 external_mini_cluster.cc:1428] Setting key 842b2580086e1b5dcb5865283e6453b7
I20260811 17:23:32.468914 5603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a62837baccfc4b83a60b873b369e27d6: Bootstrap starting.
I20260811 17:23:32.471243 5603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a62837baccfc4b83a60b873b369e27d6: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:32.472100 5603 log.cc:824] T 00000000000000000000000000000000 P a62837baccfc4b83a60b873b369e27d6: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:32.474130 5603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a62837baccfc4b83a60b873b369e27d6: No bootstrap required, opened a new log
Aug 11 17:23:32 dist-test-slave-mjjr krb5kdc[5471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:23:32.478129 5603 raft_consensus.cc:348] T 00000000000000000000000000000000 P a62837baccfc4b83a60b873b369e27d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a62837baccfc4b83a60b873b369e27d6" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41139 } }
I20260811 17:23:32.478405 5603 raft_consensus.cc:374] T 00000000000000000000000000000000 P a62837baccfc4b83a60b873b369e27d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:32.478484 5603 raft_consensus.cc:729] T 00000000000000000000000000000000 P a62837baccfc4b83a60b873b369e27d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a62837baccfc4b83a60b873b369e27d6, State: Initialized, Role: FOLLOWER
I20260811 17:23:32.479034 5603 consensus_queue.cc:260] T 00000000000000000000000000000000 P a62837baccfc4b83a60b873b369e27d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a62837baccfc4b83a60b873b369e27d6" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41139 } }
I20260811 17:23:32.479213 5603 raft_consensus.cc:388] T 00000000000000000000000000000000 P a62837baccfc4b83a60b873b369e27d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:23:32.479290 5603 raft_consensus.cc:482] T 00000000000000000000000000000000 P a62837baccfc4b83a60b873b369e27d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:23:32.479391 5603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a62837baccfc4b83a60b873b369e27d6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:32.480690 5603 raft_consensus.cc:504] T 00000000000000000000000000000000 P a62837baccfc4b83a60b873b369e27d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a62837baccfc4b83a60b873b369e27d6" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41139 } }
I20260811 17:23:32.481077 5603 leader_election.cc:302] T 00000000000000000000000000000000 P a62837baccfc4b83a60b873b369e27d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a62837baccfc4b83a60b873b369e27d6; no voters:
I20260811 17:23:32.481434 5603 leader_election.cc:288] T 00000000000000000000000000000000 P a62837baccfc4b83a60b873b369e27d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:23:32.481586 5610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a62837baccfc4b83a60b873b369e27d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:23:32.481616 5607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:32.468002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48364 (local address 127.9.157.190:41139)
0811 17:23:32.468474 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:32.468490 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:32.468529 (+ 39us) server_negotiation.cc:407] Connection header received
0811 17:23:32.469268 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:32.469288 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:32.469633 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:32.469997 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:32.470830 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:32.471982 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:32.472908 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:32.473392 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:32.473570 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:32.476186 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:32.476191 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:32.476207 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:23:32.476236 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:32.478062 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:32.478582 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:32.478588 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:32.478594 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:32.478748 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:32.479056 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:32.479060 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:32.479061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:32.479705 (+ 644us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:32.479919 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:32.480188 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 17:23:32.480547 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":144,"threads_started":1}
I20260811 17:23:32.481829 5610 raft_consensus.cc:686] T 00000000000000000000000000000000 P a62837baccfc4b83a60b873b369e27d6 [term 1 LEADER]: Becoming Leader. State: Replica: a62837baccfc4b83a60b873b369e27d6, State: Running, Role: LEADER
I20260811 17:23:32.482156 5610 consensus_queue.cc:237] T 00000000000000000000000000000000 P a62837baccfc4b83a60b873b369e27d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a62837baccfc4b83a60b873b369e27d6" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41139 } }
I20260811 17:23:32.482787 5603 sys_catalog.cc:564] T 00000000000000000000000000000000 P a62837baccfc4b83a60b873b369e27d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:23:32.483381 5613 sys_catalog.cc:455] T 00000000000000000000000000000000 P a62837baccfc4b83a60b873b369e27d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a62837baccfc4b83a60b873b369e27d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a62837baccfc4b83a60b873b369e27d6" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41139 } } }
I20260811 17:23:32.483505 5613 sys_catalog.cc:458] T 00000000000000000000000000000000 P a62837baccfc4b83a60b873b369e27d6 [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:32.483896 5611 sys_catalog.cc:455] T 00000000000000000000000000000000 P a62837baccfc4b83a60b873b369e27d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a62837baccfc4b83a60b873b369e27d6. Latest consensus state: current_term: 1 leader_uuid: "a62837baccfc4b83a60b873b369e27d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a62837baccfc4b83a60b873b369e27d6" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41139 } } }
I20260811 17:23:32.484113 5611 sys_catalog.cc:458] T 00000000000000000000000000000000 P a62837baccfc4b83a60b873b369e27d6 [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:32.484457 5617 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:23:32.488462 5617 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:23:32.495693 5617 catalog_manager.cc:1347] Generated new cluster ID: 419e7dde88ee4922903aa85df34a10db
I20260811 17:23:32.495795 5617 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:23:32.525918 5617 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:23:32.527105 5617 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:23:32.535465 5617 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a62837baccfc4b83a60b873b369e27d6: Generated new TSK 0
I20260811 17:23:32.536623 5617 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
I20260811 17:23:32.580750 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:0
--local_ip_for_outbound_sockets=127.9.157.129
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:41139
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:23:32.708508 5639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:32.708814 5639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:32.708863 5639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:32.713617 5639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:32.713724 5639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:32.713798 5639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:23:32.719821 5639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:41139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:32.721100 5639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:32.722244 5639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:32.732514 5653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:32.732506 5647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:32.732506 5649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:32.732931 5639 server_base.cc:1034] running on GCE node
I20260811 17:23:32.733441 5639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:32.734120 5639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:32.735307 5639 hybrid_clock.cc:648] HybridClock initialized: now 1786469012735263 us; error 55 us; skew 500 ppm
Aug 11 17:23:32 dist-test-slave-mjjr krb5kdc[5471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469012, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:23:32.738502 5639 init.cc:377] Logged in from keytab as kudu/127.9.157.129@KRBTEST.COM (short username kudu)
I20260811 17:23:32.739508 5639 webserver.cc:460] Webserver started at http://127.9.157.129:46407/ using document root <none> and password file <none>
I20260811 17:23:32.740159 5639 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:32.740226 5639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:32.740410 5639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:32.742584 5639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-0/data/instance:
uuid: "baacba75aec34a1588d4f028a1eb7436"
format_stamp: "Formatted at 2026-08-11 17:23:32 on dist-test-slave-mjjr"
server_key: "f87b2bf19846377c391f3144e81b3ad5"
server_key_iv: "bacefe3616b4310afcbd68d70ff51b9e"
server_key_version: "encryptionkey@0"
I20260811 17:23:32.743146 5639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance:
uuid: "baacba75aec34a1588d4f028a1eb7436"
format_stamp: "Formatted at 2026-08-11 17:23:32 on dist-test-slave-mjjr"
server_key: "f87b2bf19846377c391f3144e81b3ad5"
server_key_iv: "bacefe3616b4310afcbd68d70ff51b9e"
server_key_version: "encryptionkey@0"
I20260811 17:23:32.747259 5639 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:23:32.750072 5667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:32.751490 5639 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 17:23:32.751698 5639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "baacba75aec34a1588d4f028a1eb7436"
format_stamp: "Formatted at 2026-08-11 17:23:32 on dist-test-slave-mjjr"
server_key: "f87b2bf19846377c391f3144e81b3ad5"
server_key_iv: "bacefe3616b4310afcbd68d70ff51b9e"
server_key_version: "encryptionkey@0"
I20260811 17:23:32.751857 5639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:32.765327 5639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:32.768757 5639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:32.769009 5639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:32.769675 5639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:23:32.770588 5639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:23:32.770665 5639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:32.770740 5639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:23:32.770780 5639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:32.780776 5639 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:45071
I20260811 17:23:32.780795 5831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:45071 every 8 connection(s)
I20260811 17:23:32.781822 5639 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:23:32.788985 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 5639
I20260811 17:23:32.789160 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance
I20260811 17:23:32.789506 9846 external_mini_cluster.cc:1428] Setting key d25101dbb26c1d5613351b6ec23110ff
Aug 11 17:23:32 dist-test-slave-mjjr krb5kdc[5471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469012, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:23:32.797597 5607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:32.783950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:56391 (local address 127.9.157.190:41139)
0811 17:23:32.784087 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:32.784091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:32.785091 (+ 1000us) server_negotiation.cc:407] Connection header received
0811 17:23:32.786577 (+ 1486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:32.786580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:32.786643 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:32.786751 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:32.788160 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:32.788714 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:32.790120 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:32.790378 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:32.790478 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:32.793705 (+ 3227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:32.793709 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:32.793712 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:23:32.793740 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:32.795209 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:32.795848 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:32.795851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:32.795852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:32.795940 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:32.796326 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:32.796330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:32.796331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:32.796545 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:32.796662 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:32.797249 (+ 587us) server_negotiation.cc:299] Negotiation successful
0811 17:23:32.797410 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 17:23:32.798514 5837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:32.784395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:41139 (local address 127.9.157.129:56391)
0811 17:23:32.784934 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:32.784981 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:32.786326 (+ 1345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:32.786966 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:32.786974 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:32.787376 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:23:32.787991 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:32.788008 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:32.788870 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:32.788873 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:32.789956 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:32.789972 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:32.790557 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:32.790561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:32.790770 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:32.791537 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:23:32.791556 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:23:32.793495 (+ 1939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:23:32.795347 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:23:32.795353 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:23:32.795373 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:23:32.795727 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:23:32.796042 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:23:32.796046 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:23:32.796048 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:23:32.796213 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:23:32.796663 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:23:32.796673 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:23:32.796980 (+ 307us) client_negotiation.cc:769] Sending connection context
0811 17:23:32.797189 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 17:23:32.797578 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":119,"threads_started":1}
I20260811 17:23:32.800016 5833 heartbeater.cc:347] Connected to a master server at 127.9.157.190:41139
I20260811 17:23:32.800309 5833 heartbeater.cc:464] Registering TS with master...
I20260811 17:23:32.800899 5833 heartbeater.cc:511] Master 127.9.157.190:41139 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.157.130@KRBTEST.COM; defaulting to no policy
I20260811 17:23:32.802732 5542 ts_manager.cc:194] Registered new tserver with Master: baacba75aec34a1588d4f028a1eb7436 (127.9.157.129:45071)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.130@KRBTEST.COM" created.
I20260811 17:23:32.804220 5542 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.129@KRBTEST.COM'} at 127.9.157.129:56391
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
I20260811 17:23:32.824074 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:0
--local_ip_for_outbound_sockets=127.9.157.130
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:41139
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:23:32.949059 5844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:32.949374 5844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:32.949421 5844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:32.953987 5844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:32.954067 5844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:32.954160 5844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:23:32.960178 5844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:41139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:32.961401 5844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:32.962535 5844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:32.973098 5864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:32.973292 5844 server_base.cc:1034] running on GCE node
W20260811 17:23:32.973121 5854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:32.973089 5859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:32.973955 5844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:32.974707 5844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:32.975929 5844 hybrid_clock.cc:648] HybridClock initialized: now 1786469012975901 us; error 43 us; skew 500 ppm
Aug 11 17:23:32 dist-test-slave-mjjr krb5kdc[5471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469012, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:23:32.978984 5844 init.cc:377] Logged in from keytab as kudu/127.9.157.130@KRBTEST.COM (short username kudu)
I20260811 17:23:32.980095 5844 webserver.cc:460] Webserver started at http://127.9.157.130:35789/ using document root <none> and password file <none>
I20260811 17:23:32.980724 5844 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:32.980823 5844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:32.981073 5844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:32.983450 5844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-1/data/instance:
uuid: "b22de3bc5ca14773bf7d5c3033682c6e"
format_stamp: "Formatted at 2026-08-11 17:23:32 on dist-test-slave-mjjr"
server_key: "6104f2ae965ea7e91de3739d5fedbd5a"
server_key_iv: "eb5b54f5c4362bb66b0b8132721ecaca"
server_key_version: "encryptionkey@0"
I20260811 17:23:32.984151 5844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance:
uuid: "b22de3bc5ca14773bf7d5c3033682c6e"
format_stamp: "Formatted at 2026-08-11 17:23:32 on dist-test-slave-mjjr"
server_key: "6104f2ae965ea7e91de3739d5fedbd5a"
server_key_iv: "eb5b54f5c4362bb66b0b8132721ecaca"
server_key_version: "encryptionkey@0"
I20260811 17:23:32.988431 5844 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:23:32.991034 5871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:32.992642 5844 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:23:32.992815 5844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "b22de3bc5ca14773bf7d5c3033682c6e"
format_stamp: "Formatted at 2026-08-11 17:23:32 on dist-test-slave-mjjr"
server_key: "6104f2ae965ea7e91de3739d5fedbd5a"
server_key_iv: "eb5b54f5c4362bb66b0b8132721ecaca"
server_key_version: "encryptionkey@0"
I20260811 17:23:32.992956 5844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:33.016407 5844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:33.020462 5844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:33.020700 5844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:33.021438 5844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:23:33.022466 5844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:23:33.022547 5844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:33.022625 5844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:23:33.022676 5844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:33.033039 5844 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:41967
I20260811 17:23:33.033106 6057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:41967 every 8 connection(s)
I20260811 17:23:33.034157 5844 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:23:33.043502 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 5844
I20260811 17:23:33.043625 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance
I20260811 17:23:33.043972 9846 external_mini_cluster.cc:1428] Setting key 4b2ed884bc748dc337c959b775c79770
Aug 11 17:23:33 dist-test-slave-mjjr krb5kdc[5471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469012, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:23:33.049132 5607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:33.036241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:57205 (local address 127.9.157.190:41139)
0811 17:23:33.036403 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:33.036408 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:33.037376 (+ 968us) server_negotiation.cc:407] Connection header received
0811 17:23:33.038825 (+ 1449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:33.038828 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:33.038881 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:33.038962 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:33.040272 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.040807 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.041958 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.042175 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.042255 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:33.044962 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:33.044965 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:33.044967 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:23:33.044995 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:33.046564 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:33.047267 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:33.047269 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:33.047271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:33.047349 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:33.047800 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:33.047803 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:33.047804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:33.048093 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:33.048200 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:33.048764 (+ 564us) server_negotiation.cc:299] Negotiation successful
0811 17:23:33.048925 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 17:23:33.050019 6060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:33.036703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:41139 (local address 127.9.157.130:57205)
0811 17:23:33.037204 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:33.037252 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:33.038608 (+ 1356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:33.039143 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:33.039150 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:33.039532 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:23:33.040127 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:33.040142 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.040934 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.040937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:33.041837 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:33.041846 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.042312 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.042315 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:33.042462 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:33.043057 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:23:33.043074 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:23:33.044785 (+ 1711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:23:33.046696 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:23:33.046706 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:23:33.046729 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:23:33.047163 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:23:33.047451 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:23:33.047457 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:23:33.047462 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:23:33.047703 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:23:33.048219 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:23:33.048225 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:23:33.048497 (+ 272us) client_negotiation.cc:769] Sending connection context
0811 17:23:33.048659 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 17:23:33.048949 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":138,"threads_started":1}
I20260811 17:23:33.051285 6058 heartbeater.cc:347] Connected to a master server at 127.9.157.190:41139
I20260811 17:23:33.051554 6058 heartbeater.cc:464] Registering TS with master...
I20260811 17:23:33.052126 6058 heartbeater.cc:511] Master 127.9.157.190:41139 requested a full tablet report, sending...
I20260811 17:23:33.053273 5542 ts_manager.cc:194] Registered new tserver with Master: b22de3bc5ca14773bf7d5c3033682c6e (127.9.157.130:41967)
I20260811 17:23:33.053879 5542 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.130@KRBTEST.COM'} at 127.9.157.130:57205
WARNING: no policy specified for kudu/127.9.157.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
I20260811 17:23:33.075212 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:0
--local_ip_for_outbound_sockets=127.9.157.131
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:41139
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:38967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:23:33.198998 6064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:33.199311 6064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:33.199358 6064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:33.203665 6064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:33.203739 6064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:33.203804 6064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:23:33.209688 6064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:38967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:41139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:33.210857 6064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:33.212426 6064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:33.223384 6076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:33.223414 6080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:33.223384 6082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:33.223702 6064 server_base.cc:1034] running on GCE node
I20260811 17:23:33.224299 6064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:33.225121 6064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:33.226348 6064 hybrid_clock.cc:648] HybridClock initialized: now 1786469013226327 us; error 34 us; skew 500 ppm
Aug 11 17:23:33 dist-test-slave-mjjr krb5kdc[5471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469013, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:23:33.229427 6064 init.cc:377] Logged in from keytab as kudu/127.9.157.131@KRBTEST.COM (short username kudu)
I20260811 17:23:33.230438 6064 webserver.cc:460] Webserver started at http://127.9.157.131:41667/ using document root <none> and password file <none>
I20260811 17:23:33.231050 6064 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:33.231117 6064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:33.231305 6064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:33.233644 6064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-2/data/instance:
uuid: "abd847deb2e24fe4b1e0ce7c00d1a9b4"
format_stamp: "Formatted at 2026-08-11 17:23:33 on dist-test-slave-mjjr"
server_key: "9229565320154393ff4685446df06341"
server_key_iv: "79c533987aa355af8175ca62fe76e465"
server_key_version: "encryptionkey@0"
I20260811 17:23:33.234221 6064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance:
uuid: "abd847deb2e24fe4b1e0ce7c00d1a9b4"
format_stamp: "Formatted at 2026-08-11 17:23:33 on dist-test-slave-mjjr"
server_key: "9229565320154393ff4685446df06341"
server_key_iv: "79c533987aa355af8175ca62fe76e465"
server_key_version: "encryptionkey@0"
I20260811 17:23:33.238778 6064 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:23:33.241791 6089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:33.243193 6064 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:23:33.243309 6064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "abd847deb2e24fe4b1e0ce7c00d1a9b4"
format_stamp: "Formatted at 2026-08-11 17:23:33 on dist-test-slave-mjjr"
server_key: "9229565320154393ff4685446df06341"
server_key_iv: "79c533987aa355af8175ca62fe76e465"
server_key_version: "encryptionkey@0"
I20260811 17:23:33.243430 6064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:33.265973 6064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:33.269411 6064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:33.269593 6064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:33.270200 6064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:23:33.271086 6064 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:23:33.271139 6064 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:33.271183 6064 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:23:33.271205 6064 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:33.281301 6064 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:36667
I20260811 17:23:33.281319 6265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:36667 every 8 connection(s)
I20260811 17:23:33.282317 6064 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:23:33.284471 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 6064
I20260811 17:23:33.284641 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance
I20260811 17:23:33.284945 9846 external_mini_cluster.cc:1428] Setting key b8037c790a3f69b9d56caf6e47da496b
Aug 11 17:23:33 dist-test-slave-mjjr krb5kdc[5471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469013, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:23:33.296841 5607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:33.285435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:60197 (local address 127.9.157.190:41139)
0811 17:23:33.285785 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:33.285790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:33.285808 (+ 18us) server_negotiation.cc:407] Connection header received
0811 17:23:33.287322 (+ 1514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:33.287325 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:33.287374 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:33.287450 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:33.288718 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.289243 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.290363 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.290587 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.290665 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:33.293249 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:33.293252 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:33.293254 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:23:33.293280 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:33.294790 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:33.295290 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:33.295293 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:33.295294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:33.295366 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:33.295748 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:33.295750 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:33.295751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:33.295937 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:33.296018 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:33.296602 (+ 584us) server_negotiation.cc:299] Negotiation successful
0811 17:23:33.296709 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261}
I20260811 17:23:33.297923 6268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:33.285120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:41139 (local address 127.9.157.131:60197)
0811 17:23:33.285667 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:33.285725 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:33.287124 (+ 1399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:33.287632 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:33.287662 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:33.288052 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:23:33.288586 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:33.288600 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.289352 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.289354 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:33.290256 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:33.290266 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.290724 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.290727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:33.290877 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:33.291458 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:23:33.291476 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:23:33.293065 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:23:33.294914 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:23:33.294920 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:23:33.294934 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:23:33.295205 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:23:33.295445 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:23:33.295447 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:23:33.295449 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:23:33.295590 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:23:33.296064 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:23:33.296070 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:23:33.296341 (+ 271us) client_negotiation.cc:769] Sending connection context
0811 17:23:33.296543 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 17:23:33.296827 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":154,"threads_started":1}
I20260811 17:23:33.299202 6266 heartbeater.cc:347] Connected to a master server at 127.9.157.190:41139
I20260811 17:23:33.299492 6266 heartbeater.cc:464] Registering TS with master...
I20260811 17:23:33.300074 6266 heartbeater.cc:511] Master 127.9.157.190:41139 requested a full tablet report, sending...
I20260811 17:23:33.301167 5542 ts_manager.cc:194] Registered new tserver with Master: abd847deb2e24fe4b1e0ce7c00d1a9b4 (127.9.157.131:36667)
I20260811 17:23:33.301775 5542 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.131@KRBTEST.COM'} at 127.9.157.131:60197
I20260811 17:23:33.311465 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:23:33.321712 5607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:33.313607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48372 (local address 127.9.157.190:41139)
0811 17:23:33.313863 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:33.313870 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:33.314015 (+ 145us) server_negotiation.cc:407] Connection header received
0811 17:23:33.314126 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:33.314128 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:33.314172 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:33.314241 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:33.315035 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.315542 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.316435 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.316657 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.316741 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:33.318418 (+ 1677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:33.318422 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:33.318424 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:23:33.318451 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:33.319942 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:33.320465 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:33.320468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:33.320470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:33.320547 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:33.320822 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:33.320824 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:33.320826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:33.321051 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:33.321170 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:33.321405 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 17:23:33.321494 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157}
I20260811 17:23:33.325138 5542 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48372:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:23:33.328325 5542 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:23:33.344309 6293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:33.338908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39918 (local address 127.9.157.129:45071)
0811 17:23:33.339319 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:33.339324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:33.339342 (+ 18us) server_negotiation.cc:407] Connection header received
0811 17:23:33.339405 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:33.339409 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:33.339552 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:33.339739 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:33.340010 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.340916 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.342608 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.343474 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.343592 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:33.343827 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:33.344010 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 17:23:33.344129 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":84,"threads_started":1}
I20260811 17:23:33.344302 6288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:33.337602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:45071 (local address 127.0.0.1:39918)
0811 17:23:33.338476 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:33.338496 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:33.338591 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:33.339702 (+ 1111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:33.339706 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:33.339725 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:33.339888 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:33.339895 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.341033 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.341039 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:33.342459 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:33.342471 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.343614 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.343618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:33.343780 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:33.343822 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 17:23:33.343962 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 17:23:33.344110 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":787,"mutex_wait_us":8,"thread_start_us":90,"threads_started":1}
I20260811 17:23:33.344511 6287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:33.337911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:41967 (local address 127.0.0.1:41010)
0811 17:23:33.338986 (+ 1075us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:33.339003 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:33.339103 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:33.340125 (+ 1022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:33.340127 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:33.340148 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:33.340236 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:33.340241 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.341727 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.341731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:33.342907 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:33.342924 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.344063 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.344067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:33.344153 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:33.344177 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 17:23:33.344227 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 17:23:33.344293 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1005,"mutex_wait_us":43,"thread_start_us":118,"threads_started":1}
I20260811 17:23:33.344663 6289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:33.338314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58264 (local address 127.9.157.131:36667)
0811 17:23:33.339223 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:33.339232 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:33.339253 (+ 21us) server_negotiation.cc:407] Connection header received
0811 17:23:33.339320 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:33.339324 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:33.339491 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:33.340314 (+ 823us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:33.340503 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.341605 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.342874 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.343508 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.343640 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:33.343826 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:33.344375 (+ 549us) server_negotiation.cc:299] Negotiation successful
0811 17:23:33.344507 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":806,"thread_start_us":180,"threads_started":1}
I20260811 17:23:33.344302 6286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:33.337527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:36667 (local address 127.0.0.1:58264)
0811 17:23:33.337952 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:33.338015 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:33.338157 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:33.339670 (+ 1513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:33.339676 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:33.339703 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:33.339886 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:33.339895 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.341750 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.341753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:33.342738 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:33.342750 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.343893 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.343899 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:33.344005 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:33.344028 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 17:23:33.344093 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 17:23:33.344147 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":122,"spinlock_wait_cycles":896,"thread_start_us":114,"threads_started":1}
I20260811 17:23:33.345443 6292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:33.338430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41010 (local address 127.9.157.130:41967)
0811 17:23:33.339001 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:33.339016 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:33.339712 (+ 696us) server_negotiation.cc:407] Connection header received
0811 17:23:33.339796 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:33.339800 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:33.339996 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:33.340121 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:33.340355 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.341552 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.343042 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.343901 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.344965 (+ 1064us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:33.345139 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:33.345191 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 17:23:33.345283 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":158,"threads_started":1}
I20260811 17:23:33.348024 5736 tablet_service.cc:1467] Processing CreateTablet for tablet a974317edca540cb8fda0bf2790014b3 (DEFAULT_TABLE table=test-table [id=fa96310b8f59412f9ebbdf980d682182]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:23:33.348034 6164 tablet_service.cc:1467] Processing CreateTablet for tablet a974317edca540cb8fda0bf2790014b3 (DEFAULT_TABLE table=test-table [id=fa96310b8f59412f9ebbdf980d682182]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:23:33.348193 5953 tablet_service.cc:1467] Processing CreateTablet for tablet a974317edca540cb8fda0bf2790014b3 (DEFAULT_TABLE table=test-table [id=fa96310b8f59412f9ebbdf980d682182]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:23:33.349277 6164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a974317edca540cb8fda0bf2790014b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:33.349272 5953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a974317edca540cb8fda0bf2790014b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:33.349272 5736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a974317edca540cb8fda0bf2790014b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:33.356563 6295 tablet_bootstrap.cc:492] T a974317edca540cb8fda0bf2790014b3 P baacba75aec34a1588d4f028a1eb7436: Bootstrap starting.
I20260811 17:23:33.357913 6296 tablet_bootstrap.cc:492] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e: Bootstrap starting.
I20260811 17:23:33.358167 6297 tablet_bootstrap.cc:492] T a974317edca540cb8fda0bf2790014b3 P abd847deb2e24fe4b1e0ce7c00d1a9b4: Bootstrap starting.
I20260811 17:23:33.359293 6295 tablet_bootstrap.cc:654] T a974317edca540cb8fda0bf2790014b3 P baacba75aec34a1588d4f028a1eb7436: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:33.360237 6297 tablet_bootstrap.cc:654] T a974317edca540cb8fda0bf2790014b3 P abd847deb2e24fe4b1e0ce7c00d1a9b4: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:33.360261 6295 log.cc:824] T a974317edca540cb8fda0bf2790014b3 P baacba75aec34a1588d4f028a1eb7436: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:33.360824 6296 tablet_bootstrap.cc:654] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:33.361680 6296 log.cc:824] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:33.361678 6297 log.cc:824] T a974317edca540cb8fda0bf2790014b3 P abd847deb2e24fe4b1e0ce7c00d1a9b4: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:33.363838 6297 tablet_bootstrap.cc:492] T a974317edca540cb8fda0bf2790014b3 P abd847deb2e24fe4b1e0ce7c00d1a9b4: No bootstrap required, opened a new log
I20260811 17:23:33.364089 6297 ts_tablet_manager.cc:1397] T a974317edca540cb8fda0bf2790014b3 P abd847deb2e24fe4b1e0ce7c00d1a9b4: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 17:23:33.364176 6296 tablet_bootstrap.cc:492] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e: No bootstrap required, opened a new log
I20260811 17:23:33.364176 6295 tablet_bootstrap.cc:492] T a974317edca540cb8fda0bf2790014b3 P baacba75aec34a1588d4f028a1eb7436: No bootstrap required, opened a new log
I20260811 17:23:33.364382 6295 ts_tablet_manager.cc:1397] T a974317edca540cb8fda0bf2790014b3 P baacba75aec34a1588d4f028a1eb7436: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 17:23:33.364382 6296 ts_tablet_manager.cc:1397] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 17:23:33.367377 6297 raft_consensus.cc:348] T a974317edca540cb8fda0bf2790014b3 P abd847deb2e24fe4b1e0ce7c00d1a9b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd847deb2e24fe4b1e0ce7c00d1a9b4" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } } peers { permanent_uuid: "baacba75aec34a1588d4f028a1eb7436" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45071 } } peers { permanent_uuid: "b22de3bc5ca14773bf7d5c3033682c6e" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 41967 } }
I20260811 17:23:33.367789 6297 raft_consensus.cc:374] T a974317edca540cb8fda0bf2790014b3 P abd847deb2e24fe4b1e0ce7c00d1a9b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:33.367884 6297 raft_consensus.cc:729] T a974317edca540cb8fda0bf2790014b3 P abd847deb2e24fe4b1e0ce7c00d1a9b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abd847deb2e24fe4b1e0ce7c00d1a9b4, State: Initialized, Role: FOLLOWER
I20260811 17:23:33.368445 6297 consensus_queue.cc:260] T a974317edca540cb8fda0bf2790014b3 P abd847deb2e24fe4b1e0ce7c00d1a9b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd847deb2e24fe4b1e0ce7c00d1a9b4" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } } peers { permanent_uuid: "baacba75aec34a1588d4f028a1eb7436" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45071 } } peers { permanent_uuid: "b22de3bc5ca14773bf7d5c3033682c6e" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 41967 } }
I20260811 17:23:33.368594 6296 raft_consensus.cc:348] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd847deb2e24fe4b1e0ce7c00d1a9b4" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } } peers { permanent_uuid: "baacba75aec34a1588d4f028a1eb7436" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45071 } } peers { permanent_uuid: "b22de3bc5ca14773bf7d5c3033682c6e" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 41967 } }
I20260811 17:23:33.368952 6296 raft_consensus.cc:374] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:33.369055 6296 raft_consensus.cc:729] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b22de3bc5ca14773bf7d5c3033682c6e, State: Initialized, Role: FOLLOWER
I20260811 17:23:33.369261 6266 heartbeater.cc:503] Master 127.9.157.190:41139 was elected leader, sending a full tablet report...
I20260811 17:23:33.369163 6295 raft_consensus.cc:348] T a974317edca540cb8fda0bf2790014b3 P baacba75aec34a1588d4f028a1eb7436 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd847deb2e24fe4b1e0ce7c00d1a9b4" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } } peers { permanent_uuid: "baacba75aec34a1588d4f028a1eb7436" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45071 } } peers { permanent_uuid: "b22de3bc5ca14773bf7d5c3033682c6e" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 41967 } }
I20260811 17:23:33.369405 6295 raft_consensus.cc:374] T a974317edca540cb8fda0bf2790014b3 P baacba75aec34a1588d4f028a1eb7436 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:33.369478 6295 raft_consensus.cc:729] T a974317edca540cb8fda0bf2790014b3 P baacba75aec34a1588d4f028a1eb7436 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baacba75aec34a1588d4f028a1eb7436, State: Initialized, Role: FOLLOWER
I20260811 17:23:33.369637 6296 consensus_queue.cc:260] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd847deb2e24fe4b1e0ce7c00d1a9b4" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } } peers { permanent_uuid: "baacba75aec34a1588d4f028a1eb7436" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45071 } } peers { permanent_uuid: "b22de3bc5ca14773bf7d5c3033682c6e" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 41967 } }
I20260811 17:23:33.369853 6295 consensus_queue.cc:260] T a974317edca540cb8fda0bf2790014b3 P baacba75aec34a1588d4f028a1eb7436 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd847deb2e24fe4b1e0ce7c00d1a9b4" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } } peers { permanent_uuid: "baacba75aec34a1588d4f028a1eb7436" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45071 } } peers { permanent_uuid: "b22de3bc5ca14773bf7d5c3033682c6e" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 41967 } }
I20260811 17:23:33.369995 6297 ts_tablet_manager.cc:1428] T a974317edca540cb8fda0bf2790014b3 P abd847deb2e24fe4b1e0ce7c00d1a9b4: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 17:23:33.370635 6296 ts_tablet_manager.cc:1428] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 17:23:33.370710 6295 ts_tablet_manager.cc:1428] T a974317edca540cb8fda0bf2790014b3 P baacba75aec34a1588d4f028a1eb7436: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 17:23:33.371093 6058 heartbeater.cc:503] Master 127.9.157.190:41139 was elected leader, sending a full tablet report...
I20260811 17:23:33.372180 5833 heartbeater.cc:503] Master 127.9.157.190:41139 was elected leader, sending a full tablet report...
W20260811 17:23:33.533380 6267 tablet.cc:2369] T a974317edca540cb8fda0bf2790014b3 P abd847deb2e24fe4b1e0ce7c00d1a9b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:23:33.533389 5834 tablet.cc:2369] T a974317edca540cb8fda0bf2790014b3 P baacba75aec34a1588d4f028a1eb7436: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:23:33.535422 6059 tablet.cc:2369] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:23:33.685734 6307 raft_consensus.cc:482] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:23:33.685971 6307 raft_consensus.cc:504] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd847deb2e24fe4b1e0ce7c00d1a9b4" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } } peers { permanent_uuid: "baacba75aec34a1588d4f028a1eb7436" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45071 } } peers { permanent_uuid: "b22de3bc5ca14773bf7d5c3033682c6e" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 41967 } }
I20260811 17:23:33.687212 6307 leader_election.cc:288] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abd847deb2e24fe4b1e0ce7c00d1a9b4 (127.9.157.131:36667), baacba75aec34a1588d4f028a1eb7436 (127.9.157.129:45071)
I20260811 17:23:33.691349 6289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:33.687600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:53357 (local address 127.9.157.131:36667)
0811 17:23:33.687914 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:33.687919 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:33.687938 (+ 19us) server_negotiation.cc:407] Connection header received
0811 17:23:33.688100 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:33.688103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:33.688161 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:33.688238 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:33.688575 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.689119 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.690298 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.690721 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.690842 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:33.690902 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:33.691133 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 17:23:33.691204 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205}
I20260811 17:23:33.691378 6309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:33.687452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:36667 (local address 127.9.157.130:53357)
0811 17:23:33.687833 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:33.687850 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:33.687979 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:33.688271 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:33.688274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:33.688292 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:33.688443 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:33.688453 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.689241 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.689246 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:33.690168 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:33.690178 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.690856 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.690860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:33.690969 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:33.691023 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 17:23:33.691086 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 17:23:33.691178 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":50,"thread_start_us":121,"threads_started":1}
I20260811 17:23:33.691365 6293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:33.687720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:32777 (local address 127.9.157.129:45071)
0811 17:23:33.687854 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:33.687860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:33.688087 (+ 227us) server_negotiation.cc:407] Connection header received
0811 17:23:33.688229 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:33.688234 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:33.688289 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:33.688365 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:33.688660 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.689159 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.690332 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.690741 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.690877 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:33.690934 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:33.691166 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 17:23:33.691244 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 17:23:33.691431 6311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:33.687558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:45071 (local address 127.9.157.130:32777)
0811 17:23:33.687951 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:33.687965 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:33.688106 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:33.688390 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:33.688392 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:33.688407 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:33.688544 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:33.688552 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.689298 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.689301 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:33.690179 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:33.690190 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.690883 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.690887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:33.691022 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:33.691040 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 17:23:33.691132 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 17:23:33.691204 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":42,"thread_start_us":103,"threads_started":1}
I20260811 17:23:33.692108 5767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a974317edca540cb8fda0bf2790014b3" candidate_uuid: "b22de3bc5ca14773bf7d5c3033682c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baacba75aec34a1588d4f028a1eb7436" is_pre_election: true
I20260811 17:23:33.692107 6200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a974317edca540cb8fda0bf2790014b3" candidate_uuid: "b22de3bc5ca14773bf7d5c3033682c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abd847deb2e24fe4b1e0ce7c00d1a9b4" is_pre_election: true
I20260811 17:23:33.692461 5767 raft_consensus.cc:2454] T a974317edca540cb8fda0bf2790014b3 P baacba75aec34a1588d4f028a1eb7436 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b22de3bc5ca14773bf7d5c3033682c6e in term 0.
I20260811 17:23:33.692461 6200 raft_consensus.cc:2454] T a974317edca540cb8fda0bf2790014b3 P abd847deb2e24fe4b1e0ce7c00d1a9b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b22de3bc5ca14773bf7d5c3033682c6e in term 0.
I20260811 17:23:33.693027 5874 leader_election.cc:302] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b22de3bc5ca14773bf7d5c3033682c6e, baacba75aec34a1588d4f028a1eb7436; no voters:
I20260811 17:23:33.693293 6307 raft_consensus.cc:2781] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:23:33.693404 6307 raft_consensus.cc:482] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:23:33.693483 6307 raft_consensus.cc:3037] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:33.694571 6307 raft_consensus.cc:504] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd847deb2e24fe4b1e0ce7c00d1a9b4" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } } peers { permanent_uuid: "baacba75aec34a1588d4f028a1eb7436" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45071 } } peers { permanent_uuid: "b22de3bc5ca14773bf7d5c3033682c6e" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 41967 } }
I20260811 17:23:33.695024 6307 leader_election.cc:288] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e [CANDIDATE]: Term 1 election: Requested vote from peers abd847deb2e24fe4b1e0ce7c00d1a9b4 (127.9.157.131:36667), baacba75aec34a1588d4f028a1eb7436 (127.9.157.129:45071)
I20260811 17:23:33.695428 6200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a974317edca540cb8fda0bf2790014b3" candidate_uuid: "b22de3bc5ca14773bf7d5c3033682c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abd847deb2e24fe4b1e0ce7c00d1a9b4"
I20260811 17:23:33.695482 5767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a974317edca540cb8fda0bf2790014b3" candidate_uuid: "b22de3bc5ca14773bf7d5c3033682c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baacba75aec34a1588d4f028a1eb7436"
I20260811 17:23:33.695598 5767 raft_consensus.cc:3037] T a974317edca540cb8fda0bf2790014b3 P baacba75aec34a1588d4f028a1eb7436 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:33.695576 6200 raft_consensus.cc:3037] T a974317edca540cb8fda0bf2790014b3 P abd847deb2e24fe4b1e0ce7c00d1a9b4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:33.696969 6200 raft_consensus.cc:2454] T a974317edca540cb8fda0bf2790014b3 P abd847deb2e24fe4b1e0ce7c00d1a9b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b22de3bc5ca14773bf7d5c3033682c6e in term 1.
I20260811 17:23:33.696969 5767 raft_consensus.cc:2454] T a974317edca540cb8fda0bf2790014b3 P baacba75aec34a1588d4f028a1eb7436 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b22de3bc5ca14773bf7d5c3033682c6e in term 1.
I20260811 17:23:33.697333 5874 leader_election.cc:302] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b22de3bc5ca14773bf7d5c3033682c6e, baacba75aec34a1588d4f028a1eb7436; no voters:
I20260811 17:23:33.697543 6307 raft_consensus.cc:2781] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:23:33.697767 6307 raft_consensus.cc:686] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e [term 1 LEADER]: Becoming Leader. State: Replica: b22de3bc5ca14773bf7d5c3033682c6e, State: Running, Role: LEADER
I20260811 17:23:33.698139 6307 consensus_queue.cc:237] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd847deb2e24fe4b1e0ce7c00d1a9b4" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } } peers { permanent_uuid: "baacba75aec34a1588d4f028a1eb7436" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45071 } } peers { permanent_uuid: "b22de3bc5ca14773bf7d5c3033682c6e" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 41967 } }
I20260811 17:23:33.701751 5540 catalog_manager.cc:5495] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e reported cstate change: term changed from 0 to 1, leader changed from <none> to b22de3bc5ca14773bf7d5c3033682c6e (127.9.157.130). New cstate: current_term: 1 leader_uuid: "b22de3bc5ca14773bf7d5c3033682c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abd847deb2e24fe4b1e0ce7c00d1a9b4" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "baacba75aec34a1588d4f028a1eb7436" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b22de3bc5ca14773bf7d5c3033682c6e" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 41967 } health_report { overall_health: HEALTHY } } }
I20260811 17:23:33.799960 6292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:33.795671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41022 (local address 127.9.157.130:41967)
0811 17:23:33.795856 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:33.795862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:33.795881 (+ 19us) server_negotiation.cc:407] Connection header received
0811 17:23:33.795971 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:33.795973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:33.796033 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:33.796168 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:23:33.796474 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.797036 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.798271 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:33.798517 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:33.798632 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:33.798984 (+ 352us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:23:33.799426 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:23:33.799588 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:33.799744 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 17:23:33.799797 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 17:23:33.806247 5767 raft_consensus.cc:1261] T a974317edca540cb8fda0bf2790014b3 P baacba75aec34a1588d4f028a1eb7436 [term 1 FOLLOWER]: Refusing update from remote peer b22de3bc5ca14773bf7d5c3033682c6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:23:33.806247 6200 raft_consensus.cc:1261] T a974317edca540cb8fda0bf2790014b3 P abd847deb2e24fe4b1e0ce7c00d1a9b4 [term 1 FOLLOWER]: Refusing update from remote peer b22de3bc5ca14773bf7d5c3033682c6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:23:33.807075 6307 consensus_queue.cc:1035] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "abd847deb2e24fe4b1e0ce7c00d1a9b4" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 36667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:23:33.807381 6312 consensus_queue.cc:1035] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "baacba75aec34a1588d4f028a1eb7436" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 45071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:23:33.817370 6322 mvcc.cc:205] Tried to move back new op lower bound from 7317377080542416896 to 7317377080110206976. Current Snapshot: MvccSnapshot[applied={T|T < 7317377080542416896}]
I20260811 17:23:33.826913 5540 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48372:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:23:33.827157 5540 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48372:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:23:33.830132 5540 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a62837baccfc4b83a60b873b369e27d6: Sending DeleteTablet for 3 replicas of tablet a974317edca540cb8fda0bf2790014b3
I20260811 17:23:33.831151 5953 tablet_service.cc:1514] Processing DeleteTablet for tablet a974317edca540cb8fda0bf2790014b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:23:33 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:41010
I20260811 17:23:33.831116 6164 tablet_service.cc:1514] Processing DeleteTablet for tablet a974317edca540cb8fda0bf2790014b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:23:33 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:58264
I20260811 17:23:33.831238 5736 tablet_service.cc:1514] Processing DeleteTablet for tablet a974317edca540cb8fda0bf2790014b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:23:33 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:39918
I20260811 17:23:33.831780 6337 tablet_replica.cc:330] T a974317edca540cb8fda0bf2790014b3 P abd847deb2e24fe4b1e0ce7c00d1a9b4: stopping tablet replica
I20260811 17:23:33.832054 6338 tablet_replica.cc:330] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e: stopping tablet replica
I20260811 17:23:33.832159 6340 tablet_replica.cc:330] T a974317edca540cb8fda0bf2790014b3 P baacba75aec34a1588d4f028a1eb7436: stopping tablet replica
I20260811 17:23:33.832458 6340 raft_consensus.cc:2229] T a974317edca540cb8fda0bf2790014b3 P baacba75aec34a1588d4f028a1eb7436 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:23:33.832458 6338 raft_consensus.cc:2229] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:23:33.832826 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 5639
I20260811 17:23:33.843070 6337 raft_consensus.cc:2229] T a974317edca540cb8fda0bf2790014b3 P abd847deb2e24fe4b1e0ce7c00d1a9b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:23:33.843670 6337 raft_consensus.cc:2258] T a974317edca540cb8fda0bf2790014b3 P abd847deb2e24fe4b1e0ce7c00d1a9b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:23:33.843842 6338 raft_consensus.cc:2258] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:23:33.846004 6338 ts_tablet_manager.cc:1905] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:23:33.846203 6337 ts_tablet_manager.cc:1905] T a974317edca540cb8fda0bf2790014b3 P abd847deb2e24fe4b1e0ce7c00d1a9b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:23:33.849416 6338 ts_tablet_manager.cc:1918] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:23:33.849596 6338 log.cc:1196] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-1/wal/wals/a974317edca540cb8fda0bf2790014b3
I20260811 17:23:33.850019 6338 ts_tablet_manager.cc:1939] T a974317edca540cb8fda0bf2790014b3 P b22de3bc5ca14773bf7d5c3033682c6e: Deleting consensus metadata
I20260811 17:23:33.850000 6337 ts_tablet_manager.cc:1918] T a974317edca540cb8fda0bf2790014b3 P abd847deb2e24fe4b1e0ce7c00d1a9b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:23:33.850301 6337 log.cc:1196] T a974317edca540cb8fda0bf2790014b3 P abd847deb2e24fe4b1e0ce7c00d1a9b4: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786468743878666-9846-0/minicluster-data/ts-2/wal/wals/a974317edca540cb8fda0bf2790014b3
I20260811 17:23:33.850747 6337 ts_tablet_manager.cc:1939] T a974317edca540cb8fda0bf2790014b3 P abd847deb2e24fe4b1e0ce7c00d1a9b4: Deleting consensus metadata
W20260811 17:23:33.851130 5517 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.157.129:45071 (error 108)
I20260811 17:23:33.851617 5520 catalog_manager.cc:4841] TS b22de3bc5ca14773bf7d5c3033682c6e (127.9.157.130:41967): tablet a974317edca540cb8fda0bf2790014b3 (table test-table [id=fa96310b8f59412f9ebbdf980d682182]) successfully deleted
I20260811 17:23:33.852469 5522 catalog_manager.cc:4841] TS abd847deb2e24fe4b1e0ce7c00d1a9b4 (127.9.157.131:36667): tablet a974317edca540cb8fda0bf2790014b3 (table test-table [id=fa96310b8f59412f9ebbdf980d682182]) successfully deleted
I20260811 17:23:33.852538 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 5844
I20260811 17:23:33.855773 6343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:33.852531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:45071 (local address 127.0.0.1:39930)
0811 17:23:33.855429 (+ 2898us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:33.855571 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.129:45071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2761,"thread_start_us":64,"threads_started":1}
W20260811 17:23:33.856003 5517 catalog_manager.cc:4584] TS baacba75aec34a1588d4f028a1eb7436 (127.9.157.129:45071): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a974317edca540cb8fda0bf2790014b3: Network error: Client connection negotiation failed: client connection to 127.9.157.129:45071: connect: Connection refused (error 111)
I20260811 17:23:33.874513 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 6064
I20260811 17:23:33.894942 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 5492
2026-08-11T17:23:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (3045 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T17:23:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:23:33Z Disabled control of system clock
I20260811 17:23:33.935997 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:39681
--webserver_interface=127.9.157.190
--webserver_port=0
--builtin_ntp_servers=127.9.157.148:42239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:39681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 17:23:34.054543 6348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:34.054831 6348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:34.054869 6348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:34.058872 6348 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:23:34.058944 6348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:34.058960 6348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:34.058977 6348 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:23:34.058992 6348 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:23:34.064101 6348 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:42239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:39681
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:39681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:34.065289 6348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:34.066723 6348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:34.076277 6356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:34.076303 6357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:34.076300 6359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:34.076478 6348 server_base.cc:1034] running on GCE node
I20260811 17:23:34.077332 6348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:34.078567 6348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:34.079754 6348 hybrid_clock.cc:648] HybridClock initialized: now 1786469014079734 us; error 35 us; skew 500 ppm
I20260811 17:23:34.082016 6348 webserver.cc:460] Webserver started at http://127.9.157.190:42263/ using document root <none> and password file <none>
I20260811 17:23:34.082717 6348 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:34.082808 6348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:34.083050 6348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:34.085423 6348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "8ff0589ff6ac484ba88e73d1c2f92756"
format_stamp: "Formatted at 2026-08-11 17:23:34 on dist-test-slave-mjjr"
server_key: "a2e9d7c5ed12907f28d2f8396b40be5e"
server_key_iv: "95a3fad507939dc28771d1696814a571"
server_key_version: "encryptionkey@0"
I20260811 17:23:34.086079 6348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "8ff0589ff6ac484ba88e73d1c2f92756"
format_stamp: "Formatted at 2026-08-11 17:23:34 on dist-test-slave-mjjr"
server_key: "a2e9d7c5ed12907f28d2f8396b40be5e"
server_key_iv: "95a3fad507939dc28771d1696814a571"
server_key_version: "encryptionkey@0"
I20260811 17:23:34.090335 6348 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 17:23:34.093221 6367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:34.094720 6348 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:23:34.094995 6348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "8ff0589ff6ac484ba88e73d1c2f92756"
format_stamp: "Formatted at 2026-08-11 17:23:34 on dist-test-slave-mjjr"
server_key: "a2e9d7c5ed12907f28d2f8396b40be5e"
server_key_iv: "95a3fad507939dc28771d1696814a571"
server_key_version: "encryptionkey@0"
I20260811 17:23:34.095132 6348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:34.118163 6348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:34.118990 6348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:34.119203 6348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:34.126720 6348 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:39681
I20260811 17:23:34.126752 6445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:39681 every 8 connection(s)
I20260811 17:23:34.127804 6348 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:23:34.131034 6446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:34.133287 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 6348
I20260811 17:23:34.133453 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:23:34.133754 9846 external_mini_cluster.cc:1428] Setting key 88c3fdefc738ba5502f8d213416a9474
I20260811 17:23:34.137315 6446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ff0589ff6ac484ba88e73d1c2f92756: Bootstrap starting.
I20260811 17:23:34.139585 6446 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ff0589ff6ac484ba88e73d1c2f92756: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:34.140515 6446 log.cc:824] T 00000000000000000000000000000000 P 8ff0589ff6ac484ba88e73d1c2f92756: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:34.142621 6446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ff0589ff6ac484ba88e73d1c2f92756: No bootstrap required, opened a new log
I20260811 17:23:34.143106 6455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:34.135166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40256 (local address 127.9.157.190:39681)
0811 17:23:34.135893 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:34.135928 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:34.135992 (+ 64us) server_negotiation.cc:407] Connection header received
0811 17:23:34.136651 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:34.136668 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:34.137079 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:34.137413 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:34.137843 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:34.138854 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:34.139811 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:34.140294 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:34.140475 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:34.141336 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:34.141341 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:34.141355 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:34.141747 (+ 392us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:34.141803 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:34.141823 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:34.141943 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:34.142063 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 17:23:34.142328 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":134,"threads_started":1}
I20260811 17:23:34.146425 6446 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ff0589ff6ac484ba88e73d1c2f92756 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ff0589ff6ac484ba88e73d1c2f92756" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 39681 } }
I20260811 17:23:34.146737 6446 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ff0589ff6ac484ba88e73d1c2f92756 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:34.146808 6446 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ff0589ff6ac484ba88e73d1c2f92756 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ff0589ff6ac484ba88e73d1c2f92756, State: Initialized, Role: FOLLOWER
I20260811 17:23:34.147267 6446 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ff0589ff6ac484ba88e73d1c2f92756 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ff0589ff6ac484ba88e73d1c2f92756" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 39681 } }
I20260811 17:23:34.147414 6446 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ff0589ff6ac484ba88e73d1c2f92756 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:23:34.147496 6446 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ff0589ff6ac484ba88e73d1c2f92756 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:23:34.147614 6446 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ff0589ff6ac484ba88e73d1c2f92756 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:34.148648 6446 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ff0589ff6ac484ba88e73d1c2f92756 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ff0589ff6ac484ba88e73d1c2f92756" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 39681 } }
I20260811 17:23:34.149001 6446 leader_election.cc:302] T 00000000000000000000000000000000 P 8ff0589ff6ac484ba88e73d1c2f92756 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ff0589ff6ac484ba88e73d1c2f92756; no voters:
I20260811 17:23:34.149353 6446 leader_election.cc:288] T 00000000000000000000000000000000 P 8ff0589ff6ac484ba88e73d1c2f92756 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:23:34.149484 6457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ff0589ff6ac484ba88e73d1c2f92756 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:23:34.149747 6457 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ff0589ff6ac484ba88e73d1c2f92756 [term 1 LEADER]: Becoming Leader. State: Replica: 8ff0589ff6ac484ba88e73d1c2f92756, State: Running, Role: LEADER
I20260811 17:23:34.150089 6457 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ff0589ff6ac484ba88e73d1c2f92756 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ff0589ff6ac484ba88e73d1c2f92756" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 39681 } }
I20260811 17:23:34.150657 6446 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ff0589ff6ac484ba88e73d1c2f92756 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:23:34.152002 6458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ff0589ff6ac484ba88e73d1c2f92756 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ff0589ff6ac484ba88e73d1c2f92756" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ff0589ff6ac484ba88e73d1c2f92756" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 39681 } } }
I20260811 17:23:34.152015 6459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ff0589ff6ac484ba88e73d1c2f92756 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ff0589ff6ac484ba88e73d1c2f92756. Latest consensus state: current_term: 1 leader_uuid: "8ff0589ff6ac484ba88e73d1c2f92756" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ff0589ff6ac484ba88e73d1c2f92756" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 39681 } } }
I20260811 17:23:34.152213 6458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ff0589ff6ac484ba88e73d1c2f92756 [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:34.152226 6459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ff0589ff6ac484ba88e73d1c2f92756 [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:34.152722 6467 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:23:34.156098 6467 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:23:34.162662 6467 catalog_manager.cc:1347] Generated new cluster ID: 7bc3d5b7fb614fb093bb74416fa9c902
I20260811 17:23:34.162755 6467 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:23:34.184614 6467 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:23:34.185704 6467 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:23:34.191036 6467 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ff0589ff6ac484ba88e73d1c2f92756: Generated new TSK 0
I20260811 17:23:34.192051 6467 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:23:34.204619 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:0
--local_ip_for_outbound_sockets=127.9.157.129
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:39681
--builtin_ntp_servers=127.9.157.148:42239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 17:23:34.333277 6485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:34.333630 6485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:34.333715 6485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:34.338212 6485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:34.338321 6485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:34.338420 6485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:23:34.343802 6485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:42239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:39681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:34.345054 6485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:34.346581 6485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:34.357409 6493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:34.357580 6485 server_base.cc:1034] running on GCE node
W20260811 17:23:34.357420 6492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:34.357798 6498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:34.358314 6485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:34.358990 6485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:34.360157 6485 hybrid_clock.cc:648] HybridClock initialized: now 1786469014360138 us; error 48 us; skew 500 ppm
I20260811 17:23:34.362301 6485 webserver.cc:460] Webserver started at http://127.9.157.129:43231/ using document root <none> and password file <none>
I20260811 17:23:34.362917 6485 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:34.363010 6485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:34.363250 6485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:34.365530 6485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-0/data/instance:
uuid: "914f15c163d34f17b7146896a5fcdcab"
format_stamp: "Formatted at 2026-08-11 17:23:34 on dist-test-slave-mjjr"
server_key: "f3684e1f0ac80541966f9fbd85d33312"
server_key_iv: "fdbeb601aee31e42d83ef4440991275c"
server_key_version: "encryptionkey@0"
I20260811 17:23:34.366123 6485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance:
uuid: "914f15c163d34f17b7146896a5fcdcab"
format_stamp: "Formatted at 2026-08-11 17:23:34 on dist-test-slave-mjjr"
server_key: "f3684e1f0ac80541966f9fbd85d33312"
server_key_iv: "fdbeb601aee31e42d83ef4440991275c"
server_key_version: "encryptionkey@0"
I20260811 17:23:34.370024 6485 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:23:34.372609 6509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:34.373850 6485 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 17:23:34.374019 6485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "914f15c163d34f17b7146896a5fcdcab"
format_stamp: "Formatted at 2026-08-11 17:23:34 on dist-test-slave-mjjr"
server_key: "f3684e1f0ac80541966f9fbd85d33312"
server_key_iv: "fdbeb601aee31e42d83ef4440991275c"
server_key_version: "encryptionkey@0"
I20260811 17:23:34.374142 6485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:34.409025 6485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:34.409829 6485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:34.410054 6485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:34.410727 6485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:23:34.411900 6485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:23:34.411980 6485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:34.412055 6485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:23:34.412098 6485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:34.421418 6485 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:44547
I20260811 17:23:34.421460 6696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:44547 every 8 connection(s)
I20260811 17:23:34.422432 6485 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:23:34.423534 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 6485
I20260811 17:23:34.423707 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance
I20260811 17:23:34.424103 9846 external_mini_cluster.cc:1428] Setting key d942643520e22f6bbc45b597aff91938
I20260811 17:23:34.426493 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:0
--local_ip_for_outbound_sockets=127.9.157.130
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:39681
--builtin_ntp_servers=127.9.157.148:42239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 17:23:34.434900 6455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:34.424536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:52243 (local address 127.9.157.190:39681)
0811 17:23:34.424703 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:34.424709 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:34.426180 (+ 1471us) server_negotiation.cc:407] Connection header received
0811 17:23:34.428415 (+ 2235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:34.428419 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:34.428498 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:34.428606 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:34.429792 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:34.430444 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:34.432463 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:34.432686 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:34.432774 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:34.434366 (+ 1592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:34.434369 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:34.434371 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:34.434388 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:34.434409 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:34.434419 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:34.434474 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:34.434707 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 17:23:34.434770 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 17:23:34.436408 6702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:34.425114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:39681 (local address 127.9.157.129:52243)
0811 17:23:34.426010 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:34.426069 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:34.428154 (+ 2085us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:34.428852 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:34.428859 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:34.429055 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:23:34.429622 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:34.429637 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:34.430613 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:34.430620 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:34.432316 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:34.432336 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:34.432893 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:34.432900 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:34.433191 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:34.433962 (+ 771us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:23:34.433965 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:23:34.434038 (+ 73us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:23:34.434158 (+ 120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:23:34.434508 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:23:34.434522 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:23:34.434531 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 17:23:34.434696 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 17:23:34.435100 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":155,"threads_started":1}
I20260811 17:23:34.437806 6698 heartbeater.cc:347] Connected to a master server at 127.9.157.190:39681
I20260811 17:23:34.438133 6698 heartbeater.cc:464] Registering TS with master...
I20260811 17:23:34.438918 6698 heartbeater.cc:511] Master 127.9.157.190:39681 requested a full tablet report, sending...
I20260811 17:23:34.440683 6393 ts_manager.cc:194] Registered new tserver with Master: 914f15c163d34f17b7146896a5fcdcab (127.9.157.129:44547)
I20260811 17:23:34.441892 6393 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.157.129:52243
W20260811 17:23:34.550143 6703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:34.550514 6703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:34.550593 6703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:34.555091 6703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:34.555189 6703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:34.555281 6703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:23:34.560388 6703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:42239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:39681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:34.561705 6703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:34.563387 6703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:34.574088 6710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:34.574057 6712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:34.574044 6716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:34.574425 6703 server_base.cc:1034] running on GCE node
I20260811 17:23:34.575042 6703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:34.575840 6703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:34.577031 6703 hybrid_clock.cc:648] HybridClock initialized: now 1786469014577010 us; error 31 us; skew 500 ppm
I20260811 17:23:34.579208 6703 webserver.cc:460] Webserver started at http://127.9.157.130:38929/ using document root <none> and password file <none>
I20260811 17:23:34.579954 6703 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:34.580029 6703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:34.580296 6703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:34.582552 6703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-1/data/instance:
uuid: "929b85ca20a84de2bc8d35d9237be147"
format_stamp: "Formatted at 2026-08-11 17:23:34 on dist-test-slave-mjjr"
server_key: "8894d25ee2418eb19aa708de98beb2c1"
server_key_iv: "fb7bd9bc4356d227df5bbdb80861ce57"
server_key_version: "encryptionkey@0"
I20260811 17:23:34.583179 6703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance:
uuid: "929b85ca20a84de2bc8d35d9237be147"
format_stamp: "Formatted at 2026-08-11 17:23:34 on dist-test-slave-mjjr"
server_key: "8894d25ee2418eb19aa708de98beb2c1"
server_key_iv: "fb7bd9bc4356d227df5bbdb80861ce57"
server_key_version: "encryptionkey@0"
I20260811 17:23:34.587325 6703 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 17:23:34.590011 6722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:34.591324 6703 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 17:23:34.591482 6703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "929b85ca20a84de2bc8d35d9237be147"
format_stamp: "Formatted at 2026-08-11 17:23:34 on dist-test-slave-mjjr"
server_key: "8894d25ee2418eb19aa708de98beb2c1"
server_key_iv: "fb7bd9bc4356d227df5bbdb80861ce57"
server_key_version: "encryptionkey@0"
I20260811 17:23:34.591609 6703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:34.608932 6703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:34.609746 6703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:34.609972 6703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:34.610663 6703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:23:34.611766 6703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:23:34.611833 6703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:34.611909 6703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:23:34.611956 6703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:34.621243 6703 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:37877
I20260811 17:23:34.621323 6915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:37877 every 8 connection(s)
I20260811 17:23:34.622296 6703 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:23:34.625831 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 6703
I20260811 17:23:34.625979 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance
I20260811 17:23:34.626317 9846 external_mini_cluster.cc:1428] Setting key a2bef874c86ba49bb08d22f4b29498eb
I20260811 17:23:34.628589 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:0
--local_ip_for_outbound_sockets=127.9.157.131
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:39681
--builtin_ntp_servers=127.9.157.148:42239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 17:23:34.633502 6455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:34.624304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:36277 (local address 127.9.157.190:39681)
0811 17:23:34.624472 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:34.624478 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:34.625715 (+ 1237us) server_negotiation.cc:407] Connection header received
0811 17:23:34.627608 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:34.627611 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:34.627707 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:34.627810 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:34.628902 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:34.629740 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:34.631247 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:34.631466 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:34.631548 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:34.632980 (+ 1432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:34.632983 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:34.632985 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:34.632999 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:34.633019 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:34.633028 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:34.633108 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:34.633267 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 17:23:34.633337 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 17:23:34.634430 6923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:34.624800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:39681 (local address 127.9.157.130:36277)
0811 17:23:34.625520 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:34.625586 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:34.627377 (+ 1791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:34.628061 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:34.628069 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:34.628163 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:23:34.628677 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:34.628690 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:34.629893 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:34.629896 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:34.631095 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:34.631109 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:34.631669 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:34.631676 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:34.631925 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:34.632676 (+ 751us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:23:34.632678 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:23:34.632721 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:23:34.632794 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:23:34.633110 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:23:34.633123 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:23:34.633132 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 17:23:34.633267 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 17:23:34.633586 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":126,"threads_started":1}
I20260811 17:23:34.635676 6917 heartbeater.cc:347] Connected to a master server at 127.9.157.190:39681
I20260811 17:23:34.635964 6917 heartbeater.cc:464] Registering TS with master...
I20260811 17:23:34.636576 6917 heartbeater.cc:511] Master 127.9.157.190:39681 requested a full tablet report, sending...
I20260811 17:23:34.637796 6393 ts_manager.cc:194] Registered new tserver with Master: 929b85ca20a84de2bc8d35d9237be147 (127.9.157.130:37877)
I20260811 17:23:34.638726 6393 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.157.130:36277
W20260811 17:23:34.757597 6924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:34.757929 6924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:34.757972 6924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:34.763195 6924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:34.763293 6924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:34.763360 6924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:23:34.768970 6924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:42239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:39681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:34.770371 6924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:34.771904 6924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:23:34.782720 6924 server_base.cc:1034] running on GCE node
W20260811 17:23:34.782655 6937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:34.782651 6934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:34.782646 6931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:34.783440 6924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:34.784250 6924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:34.785461 6924 hybrid_clock.cc:648] HybridClock initialized: now 1786469014785441 us; error 36 us; skew 500 ppm
I20260811 17:23:34.787810 6924 webserver.cc:460] Webserver started at http://127.9.157.131:34097/ using document root <none> and password file <none>
I20260811 17:23:34.788473 6924 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:34.788548 6924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:34.788827 6924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:34.791364 6924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-2/data/instance:
uuid: "0c2107b8a7f545b8868e3ec41044cba8"
format_stamp: "Formatted at 2026-08-11 17:23:34 on dist-test-slave-mjjr"
server_key: "1e1d71374f20dedd47878812ee3ef5d7"
server_key_iv: "cffff5f53c861b9404439d035750a2f6"
server_key_version: "encryptionkey@0"
I20260811 17:23:34.792069 6924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance:
uuid: "0c2107b8a7f545b8868e3ec41044cba8"
format_stamp: "Formatted at 2026-08-11 17:23:34 on dist-test-slave-mjjr"
server_key: "1e1d71374f20dedd47878812ee3ef5d7"
server_key_iv: "cffff5f53c861b9404439d035750a2f6"
server_key_version: "encryptionkey@0"
I20260811 17:23:34.796231 6924 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 17:23:34.799129 6946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:34.800509 6924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:23:34.800665 6924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "0c2107b8a7f545b8868e3ec41044cba8"
format_stamp: "Formatted at 2026-08-11 17:23:34 on dist-test-slave-mjjr"
server_key: "1e1d71374f20dedd47878812ee3ef5d7"
server_key_iv: "cffff5f53c861b9404439d035750a2f6"
server_key_version: "encryptionkey@0"
I20260811 17:23:34.800793 6924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:34.813124 6924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:34.813958 6924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:34.814186 6924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:34.814880 6924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:23:34.816112 6924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:23:34.816190 6924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:34.816268 6924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:23:34.816309 6924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:34.826208 6924 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:43403
I20260811 17:23:34.826236 7133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:43403 every 8 connection(s)
I20260811 17:23:34.827241 6924 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:23:34.827533 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 6924
I20260811 17:23:34.827688 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance
I20260811 17:23:34.828049 9846 external_mini_cluster.cc:1428] Setting key 34375b1d650af4f76dada238c414dffd
I20260811 17:23:34.837447 6455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:34.829403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:36341 (local address 127.9.157.190:39681)
0811 17:23:34.829570 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:34.829574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:34.830888 (+ 1314us) server_negotiation.cc:407] Connection header received
0811 17:23:34.832512 (+ 1624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:34.832516 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:34.832578 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:34.832662 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:34.833633 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:34.834186 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:34.835390 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:34.835619 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:34.835758 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:34.836919 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:34.836923 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:34.836926 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:34.836941 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:34.836962 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:34.836977 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:34.837054 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:34.837211 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 17:23:34.837281 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 17:23:34.838393 7138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:34.829940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:39681 (local address 127.9.157.131:36341)
0811 17:23:34.830742 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:34.830788 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:34.832279 (+ 1491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:34.832855 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:34.832863 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:34.832961 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:23:34.833481 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:34.833495 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:34.834338 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:34.834341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:34.835252 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:34.835262 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:34.835809 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:34.835814 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:34.836009 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:34.836639 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:23:34.836640 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:23:34.836684 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:23:34.836755 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:23:34.837057 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:23:34.837070 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:23:34.837077 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 17:23:34.837212 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 17:23:34.837513 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":144,"threads_started":1}
I20260811 17:23:34.839702 7134 heartbeater.cc:347] Connected to a master server at 127.9.157.190:39681
I20260811 17:23:34.839953 7134 heartbeater.cc:464] Registering TS with master...
I20260811 17:23:34.840515 7134 heartbeater.cc:511] Master 127.9.157.190:39681 requested a full tablet report, sending...
I20260811 17:23:34.841631 6393 ts_manager.cc:194] Registered new tserver with Master: 0c2107b8a7f545b8868e3ec41044cba8 (127.9.157.131:43403)
I20260811 17:23:34.842221 6393 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.157.131:36341
I20260811 17:23:34.842487 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:23:34.848567 6455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:34.844863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40266 (local address 127.9.157.190:39681)
0811 17:23:34.844967 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:34.844971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:34.845177 (+ 206us) server_negotiation.cc:407] Connection header received
0811 17:23:34.845278 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:34.845280 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:34.845316 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:34.845394 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:34.845673 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:34.846200 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:34.847028 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:34.847241 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:34.847326 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:34.848178 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:34.848180 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:34.848182 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:34.848194 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:34.848211 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:34.848219 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:34.848317 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:34.848420 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 17:23:34.848455 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 17:23:34.851562 6393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40266:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:23:34.854758 6393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:23:34.870564 7151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:34.863683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:43403 (local address 127.0.0.1:37430)
0811 17:23:34.864562 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:34.864575 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:34.864663 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:34.866004 (+ 1341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:34.866007 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:34.866026 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:34.866166 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:34.866174 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:34.867550 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:34.867553 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:34.868709 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:34.868719 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:34.869788 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:34.869792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:34.870019 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:34.870084 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 17:23:34.870201 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 17:23:34.870383 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":814,"mutex_wait_us":90,"thread_start_us":122,"threads_started":1}
I20260811 17:23:34.870565 7154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:34.863563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:44547 (local address 127.0.0.1:34878)
0811 17:23:34.864124 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:34.864172 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:34.864307 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:34.865609 (+ 1302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:34.865616 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:34.865653 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:34.865914 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:34.865925 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:34.867406 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:34.867409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:34.868675 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:34.868687 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:34.869789 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:34.869792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:34.870013 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:34.870100 (+ 87us) client_negotiation.cc:769] Sending connection context
0811 17:23:34.870221 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 17:23:34.870374 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":190,"spinlock_wait_cycles":5760,"thread_start_us":125,"threads_started":1}
I20260811 17:23:34.870611 7156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:34.863720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34878 (local address 127.9.157.129:44547)
0811 17:23:34.865043 (+ 1323us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:34.865055 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:34.865086 (+ 31us) server_negotiation.cc:407] Connection header received
0811 17:23:34.865177 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:34.865182 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:34.865407 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:34.865789 (+ 382us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:34.866042 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:34.867284 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:34.868814 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:34.869653 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:34.869775 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:34.869993 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:34.870261 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 17:23:34.870373 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1204,"thread_start_us":124,"threads_started":1}
I20260811 17:23:34.871292 7157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:34.864455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37430 (local address 127.9.157.131:43403)
0811 17:23:34.865472 (+ 1017us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:34.865486 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:34.865514 (+ 28us) server_negotiation.cc:407] Connection header received
0811 17:23:34.865595 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:34.865601 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:34.865799 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:34.866015 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:34.866309 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:34.867438 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:34.868872 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:34.869653 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:34.869775 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:34.869993 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:34.870996 (+ 1003us) server_negotiation.cc:299] Negotiation successful
0811 17:23:34.871119 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":894,"thread_start_us":144,"threads_started":1}
I20260811 17:23:34.871572 7160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:34.864514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:37877 (local address 127.0.0.1:37862)
0811 17:23:34.865083 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:34.865096 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:34.865197 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:34.865609 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:34.865616 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:34.865654 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:34.866122 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:34.866133 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:34.867130 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:34.867136 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:34.868675 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:34.868687 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:34.871167 (+ 2480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:34.871171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:34.871261 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:34.871285 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 17:23:34.871352 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 17:23:34.871433 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"mutex_wait_us":414,"spinlock_wait_cycles":7040,"thread_start_us":94,"threads_started":1}
I20260811 17:23:34.871714 7162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:34.864556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37862 (local address 127.9.157.130:37877)
0811 17:23:34.864935 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:34.864945 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:34.865194 (+ 249us) server_negotiation.cc:407] Connection header received
0811 17:23:34.865301 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:34.865306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:34.865457 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:34.865788 (+ 331us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:34.866231 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:34.867005 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:34.870107 (+ 3102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:34.871055 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:34.871166 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:34.871363 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:34.871423 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 17:23:34.871518 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":114,"threads_started":1}
I20260811 17:23:34.874143 7017 tablet_service.cc:1467] Processing CreateTablet for tablet f2890b982efe4bc2a8a9db6de2a22620 (DEFAULT_TABLE table=test-table [id=85113cc8c80743c5ad22f2a1470e44a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:23:34.874131 6594 tablet_service.cc:1467] Processing CreateTablet for tablet f2890b982efe4bc2a8a9db6de2a22620 (DEFAULT_TABLE table=test-table [id=85113cc8c80743c5ad22f2a1470e44a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:23:34.875154 7017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2890b982efe4bc2a8a9db6de2a22620. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:34.875154 6594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2890b982efe4bc2a8a9db6de2a22620. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:34.880453 6805 tablet_service.cc:1467] Processing CreateTablet for tablet f2890b982efe4bc2a8a9db6de2a22620 (DEFAULT_TABLE table=test-table [id=85113cc8c80743c5ad22f2a1470e44a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:23:34.881788 6805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2890b982efe4bc2a8a9db6de2a22620. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:34.882761 7163 tablet_bootstrap.cc:492] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab: Bootstrap starting.
I20260811 17:23:34.883245 7164 tablet_bootstrap.cc:492] T f2890b982efe4bc2a8a9db6de2a22620 P 0c2107b8a7f545b8868e3ec41044cba8: Bootstrap starting.
I20260811 17:23:34.885486 7163 tablet_bootstrap.cc:654] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:34.886301 7163 log.cc:824] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:34.888244 7164 tablet_bootstrap.cc:654] T f2890b982efe4bc2a8a9db6de2a22620 P 0c2107b8a7f545b8868e3ec41044cba8: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:34.888940 7163 tablet_bootstrap.cc:492] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab: No bootstrap required, opened a new log
I20260811 17:23:34.889200 7163 ts_tablet_manager.cc:1397] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 17:23:34.889168 7164 log.cc:824] T f2890b982efe4bc2a8a9db6de2a22620 P 0c2107b8a7f545b8868e3ec41044cba8: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:34.889649 7169 tablet_bootstrap.cc:492] T f2890b982efe4bc2a8a9db6de2a22620 P 929b85ca20a84de2bc8d35d9237be147: Bootstrap starting.
I20260811 17:23:34.891323 7164 tablet_bootstrap.cc:492] T f2890b982efe4bc2a8a9db6de2a22620 P 0c2107b8a7f545b8868e3ec41044cba8: No bootstrap required, opened a new log
I20260811 17:23:34.891549 7164 ts_tablet_manager.cc:1397] T f2890b982efe4bc2a8a9db6de2a22620 P 0c2107b8a7f545b8868e3ec41044cba8: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.004s
I20260811 17:23:34.892143 7169 tablet_bootstrap.cc:654] T f2890b982efe4bc2a8a9db6de2a22620 P 929b85ca20a84de2bc8d35d9237be147: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:34.893074 7169 log.cc:824] T f2890b982efe4bc2a8a9db6de2a22620 P 929b85ca20a84de2bc8d35d9237be147: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:34.893869 7163 raft_consensus.cc:348] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "914f15c163d34f17b7146896a5fcdcab" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 44547 } } peers { permanent_uuid: "0c2107b8a7f545b8868e3ec41044cba8" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 43403 } } peers { permanent_uuid: "929b85ca20a84de2bc8d35d9237be147" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37877 } }
I20260811 17:23:34.894203 7163 raft_consensus.cc:374] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:34.894264 7163 raft_consensus.cc:729] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 914f15c163d34f17b7146896a5fcdcab, State: Initialized, Role: FOLLOWER
I20260811 17:23:34.894767 7163 consensus_queue.cc:260] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "914f15c163d34f17b7146896a5fcdcab" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 44547 } } peers { permanent_uuid: "0c2107b8a7f545b8868e3ec41044cba8" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 43403 } } peers { permanent_uuid: "929b85ca20a84de2bc8d35d9237be147" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37877 } }
I20260811 17:23:34.895691 7164 raft_consensus.cc:348] T f2890b982efe4bc2a8a9db6de2a22620 P 0c2107b8a7f545b8868e3ec41044cba8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "914f15c163d34f17b7146896a5fcdcab" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 44547 } } peers { permanent_uuid: "0c2107b8a7f545b8868e3ec41044cba8" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 43403 } } peers { permanent_uuid: "929b85ca20a84de2bc8d35d9237be147" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37877 } }
I20260811 17:23:34.895972 7164 raft_consensus.cc:374] T f2890b982efe4bc2a8a9db6de2a22620 P 0c2107b8a7f545b8868e3ec41044cba8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:34.896040 7164 raft_consensus.cc:729] T f2890b982efe4bc2a8a9db6de2a22620 P 0c2107b8a7f545b8868e3ec41044cba8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c2107b8a7f545b8868e3ec41044cba8, State: Initialized, Role: FOLLOWER
I20260811 17:23:34.896232 7163 ts_tablet_manager.cc:1428] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 17:23:34.896533 7164 consensus_queue.cc:260] T f2890b982efe4bc2a8a9db6de2a22620 P 0c2107b8a7f545b8868e3ec41044cba8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "914f15c163d34f17b7146896a5fcdcab" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 44547 } } peers { permanent_uuid: "0c2107b8a7f545b8868e3ec41044cba8" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 43403 } } peers { permanent_uuid: "929b85ca20a84de2bc8d35d9237be147" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37877 } }
I20260811 17:23:34.896714 6698 heartbeater.cc:503] Master 127.9.157.190:39681 was elected leader, sending a full tablet report...
I20260811 17:23:34.896771 7169 tablet_bootstrap.cc:492] T f2890b982efe4bc2a8a9db6de2a22620 P 929b85ca20a84de2bc8d35d9237be147: No bootstrap required, opened a new log
I20260811 17:23:34.897013 7169 ts_tablet_manager.cc:1397] T f2890b982efe4bc2a8a9db6de2a22620 P 929b85ca20a84de2bc8d35d9237be147: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.002s
I20260811 17:23:34.897238 7134 heartbeater.cc:503] Master 127.9.157.190:39681 was elected leader, sending a full tablet report...
I20260811 17:23:34.898533 7164 ts_tablet_manager.cc:1428] T f2890b982efe4bc2a8a9db6de2a22620 P 0c2107b8a7f545b8868e3ec41044cba8: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 17:23:34.901552 7169 raft_consensus.cc:348] T f2890b982efe4bc2a8a9db6de2a22620 P 929b85ca20a84de2bc8d35d9237be147 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "914f15c163d34f17b7146896a5fcdcab" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 44547 } } peers { permanent_uuid: "0c2107b8a7f545b8868e3ec41044cba8" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 43403 } } peers { permanent_uuid: "929b85ca20a84de2bc8d35d9237be147" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37877 } }
I20260811 17:23:34.901870 7169 raft_consensus.cc:374] T f2890b982efe4bc2a8a9db6de2a22620 P 929b85ca20a84de2bc8d35d9237be147 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:34.901939 7169 raft_consensus.cc:729] T f2890b982efe4bc2a8a9db6de2a22620 P 929b85ca20a84de2bc8d35d9237be147 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 929b85ca20a84de2bc8d35d9237be147, State: Initialized, Role: FOLLOWER
I20260811 17:23:34.902400 7169 consensus_queue.cc:260] T f2890b982efe4bc2a8a9db6de2a22620 P 929b85ca20a84de2bc8d35d9237be147 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "914f15c163d34f17b7146896a5fcdcab" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 44547 } } peers { permanent_uuid: "0c2107b8a7f545b8868e3ec41044cba8" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 43403 } } peers { permanent_uuid: "929b85ca20a84de2bc8d35d9237be147" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37877 } }
I20260811 17:23:34.903162 6917 heartbeater.cc:503] Master 127.9.157.190:39681 was elected leader, sending a full tablet report...
I20260811 17:23:34.903481 7169 ts_tablet_manager.cc:1428] T f2890b982efe4bc2a8a9db6de2a22620 P 929b85ca20a84de2bc8d35d9237be147: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
W20260811 17:23:34.923722 6699 tablet.cc:2369] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:23:35.078866 7136 tablet.cc:2369] T f2890b982efe4bc2a8a9db6de2a22620 P 0c2107b8a7f545b8868e3ec41044cba8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:23:35.123592 6922 tablet.cc:2369] T f2890b982efe4bc2a8a9db6de2a22620 P 929b85ca20a84de2bc8d35d9237be147: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:23:35.206944 7175 raft_consensus.cc:482] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:23:35.207190 7175 raft_consensus.cc:504] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "914f15c163d34f17b7146896a5fcdcab" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 44547 } } peers { permanent_uuid: "0c2107b8a7f545b8868e3ec41044cba8" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 43403 } } peers { permanent_uuid: "929b85ca20a84de2bc8d35d9237be147" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37877 } }
I20260811 17:23:35.208490 7175 leader_election.cc:288] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c2107b8a7f545b8868e3ec41044cba8 (127.9.157.131:43403), 929b85ca20a84de2bc8d35d9237be147 (127.9.157.130:37877)
I20260811 17:23:35.212901 7157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:35.208995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:52353 (local address 127.9.157.131:43403)
0811 17:23:35.209231 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:35.209236 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:35.209261 (+ 25us) server_negotiation.cc:407] Connection header received
0811 17:23:35.209435 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:35.209439 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:35.209512 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:35.209595 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:35.209967 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:35.210541 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:35.211856 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:35.212277 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:35.212422 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:35.212477 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:35.212702 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 17:23:35.212780 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260811 17:23:35.212894 7179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:35.208821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:37877 (local address 127.9.157.129:33809)
0811 17:23:35.209459 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:35.209474 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:35.209585 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:35.209837 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:35.209839 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:35.209855 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:35.210021 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:35.210029 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:35.210881 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:35.210885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:35.211699 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:35.211709 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:35.212389 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:35.212394 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:35.212506 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:35.212537 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 17:23:35.212603 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 17:23:35.212700 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":232,"thread_start_us":92,"threads_started":1}
I20260811 17:23:35.212878 7162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:35.208960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:33809 (local address 127.9.157.130:37877)
0811 17:23:35.209107 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:35.209112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:35.209596 (+ 484us) server_negotiation.cc:407] Connection header received
0811 17:23:35.209663 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:35.209665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:35.209722 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:35.209811 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:35.210170 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:35.210754 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:35.211841 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:35.212235 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:35.212371 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:35.212436 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:35.212652 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 17:23:35.212737 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 17:23:35.212903 7178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:35.208821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:43403 (local address 127.9.157.129:52353)
0811 17:23:35.209189 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:35.209202 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:35.209306 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:35.209630 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:35.209636 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:35.209656 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:35.209839 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:35.209847 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:35.210672 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:35.210675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:35.211633 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:35.211697 (+ 64us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:35.212435 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:35.212440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:35.212546 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:35.212565 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 17:23:35.212664 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 17:23:35.212734 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":105,"threads_started":1}
I20260811 17:23:35.213634 6838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2890b982efe4bc2a8a9db6de2a22620" candidate_uuid: "914f15c163d34f17b7146896a5fcdcab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "929b85ca20a84de2bc8d35d9237be147" is_pre_election: true
I20260811 17:23:35.213797 7050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2890b982efe4bc2a8a9db6de2a22620" candidate_uuid: "914f15c163d34f17b7146896a5fcdcab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c2107b8a7f545b8868e3ec41044cba8" is_pre_election: true
I20260811 17:23:35.214018 6838 raft_consensus.cc:2454] T f2890b982efe4bc2a8a9db6de2a22620 P 929b85ca20a84de2bc8d35d9237be147 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 914f15c163d34f17b7146896a5fcdcab in term 0.
I20260811 17:23:35.214126 7050 raft_consensus.cc:2454] T f2890b982efe4bc2a8a9db6de2a22620 P 0c2107b8a7f545b8868e3ec41044cba8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 914f15c163d34f17b7146896a5fcdcab in term 0.
I20260811 17:23:35.214655 6517 leader_election.cc:302] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 914f15c163d34f17b7146896a5fcdcab, 929b85ca20a84de2bc8d35d9237be147; no voters:
I20260811 17:23:35.214962 7175 raft_consensus.cc:2781] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:23:35.215085 7175 raft_consensus.cc:482] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:23:35.215148 7175 raft_consensus.cc:3037] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:35.216434 7175 raft_consensus.cc:504] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "914f15c163d34f17b7146896a5fcdcab" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 44547 } } peers { permanent_uuid: "0c2107b8a7f545b8868e3ec41044cba8" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 43403 } } peers { permanent_uuid: "929b85ca20a84de2bc8d35d9237be147" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37877 } }
I20260811 17:23:35.216885 7175 leader_election.cc:288] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab [CANDIDATE]: Term 1 election: Requested vote from peers 0c2107b8a7f545b8868e3ec41044cba8 (127.9.157.131:43403), 929b85ca20a84de2bc8d35d9237be147 (127.9.157.130:37877)
I20260811 17:23:35.217326 7050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2890b982efe4bc2a8a9db6de2a22620" candidate_uuid: "914f15c163d34f17b7146896a5fcdcab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c2107b8a7f545b8868e3ec41044cba8"
I20260811 17:23:35.217389 6838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2890b982efe4bc2a8a9db6de2a22620" candidate_uuid: "914f15c163d34f17b7146896a5fcdcab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "929b85ca20a84de2bc8d35d9237be147"
I20260811 17:23:35.217485 7050 raft_consensus.cc:3037] T f2890b982efe4bc2a8a9db6de2a22620 P 0c2107b8a7f545b8868e3ec41044cba8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:35.217549 6838 raft_consensus.cc:3037] T f2890b982efe4bc2a8a9db6de2a22620 P 929b85ca20a84de2bc8d35d9237be147 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:35.218665 7050 raft_consensus.cc:2454] T f2890b982efe4bc2a8a9db6de2a22620 P 0c2107b8a7f545b8868e3ec41044cba8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 914f15c163d34f17b7146896a5fcdcab in term 1.
I20260811 17:23:35.218688 6838 raft_consensus.cc:2454] T f2890b982efe4bc2a8a9db6de2a22620 P 929b85ca20a84de2bc8d35d9237be147 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 914f15c163d34f17b7146896a5fcdcab in term 1.
I20260811 17:23:35.219074 6517 leader_election.cc:302] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 914f15c163d34f17b7146896a5fcdcab, 929b85ca20a84de2bc8d35d9237be147; no voters:
I20260811 17:23:35.219304 7175 raft_consensus.cc:2781] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:23:35.219537 7175 raft_consensus.cc:686] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab [term 1 LEADER]: Becoming Leader. State: Replica: 914f15c163d34f17b7146896a5fcdcab, State: Running, Role: LEADER
I20260811 17:23:35.219959 7175 consensus_queue.cc:237] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "914f15c163d34f17b7146896a5fcdcab" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 44547 } } peers { permanent_uuid: "0c2107b8a7f545b8868e3ec41044cba8" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 43403 } } peers { permanent_uuid: "929b85ca20a84de2bc8d35d9237be147" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37877 } }
I20260811 17:23:35.223395 6392 catalog_manager.cc:5495] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab reported cstate change: term changed from 0 to 1, leader changed from <none> to 914f15c163d34f17b7146896a5fcdcab (127.9.157.129). New cstate: current_term: 1 leader_uuid: "914f15c163d34f17b7146896a5fcdcab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "914f15c163d34f17b7146896a5fcdcab" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 44547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c2107b8a7f545b8868e3ec41044cba8" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 43403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "929b85ca20a84de2bc8d35d9237be147" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37877 } health_report { overall_health: UNKNOWN } } }
I20260811 17:23:35.239075 7156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:35.235426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34880 (local address 127.9.157.129:44547)
0811 17:23:35.235576 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:35.235580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:35.235596 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:23:35.235725 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:35.235728 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:35.235788 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:35.235893 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:23:35.236130 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:35.236647 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:35.237494 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:35.237711 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:35.237795 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:35.238169 (+ 374us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:23:35.238603 (+ 434us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:23:35.238713 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:35.238813 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 17:23:35.238894 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 17:23:35.245779 7050 raft_consensus.cc:1261] T f2890b982efe4bc2a8a9db6de2a22620 P 0c2107b8a7f545b8868e3ec41044cba8 [term 1 FOLLOWER]: Refusing update from remote peer 914f15c163d34f17b7146896a5fcdcab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:23:35.245779 6838 raft_consensus.cc:1261] T f2890b982efe4bc2a8a9db6de2a22620 P 929b85ca20a84de2bc8d35d9237be147 [term 1 FOLLOWER]: Refusing update from remote peer 914f15c163d34f17b7146896a5fcdcab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:23:35.246645 7180 consensus_queue.cc:1035] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c2107b8a7f545b8868e3ec41044cba8" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 43403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:23:35.246850 7175 consensus_queue.cc:1035] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab [LEADER]: Connected to new peer: Peer: permanent_uuid: "929b85ca20a84de2bc8d35d9237be147" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 37877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:23:35.256232 7188 mvcc.cc:205] Tried to move back new op lower bound from 7317377086438420480 to 7317377086344093696. Current Snapshot: MvccSnapshot[applied={T|T < 7317377086438420480}]
I20260811 17:23:35.256732 7189 mvcc.cc:205] Tried to move back new op lower bound from 7317377086438420480 to 7317377086344093696. Current Snapshot: MvccSnapshot[applied={T|T < 7317377086438420480}]
I20260811 17:23:35.264820 6391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:23:35.265048 6391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:23:35.267858 6391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8ff0589ff6ac484ba88e73d1c2f92756: Sending DeleteTablet for 3 replicas of tablet f2890b982efe4bc2a8a9db6de2a22620
I20260811 17:23:35.268748 6594 tablet_service.cc:1514] Processing DeleteTablet for tablet f2890b982efe4bc2a8a9db6de2a22620 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:23:35 UTC) from {username='slave'} at 127.0.0.1:34878
I20260811 17:23:35.268863 7017 tablet_service.cc:1514] Processing DeleteTablet for tablet f2890b982efe4bc2a8a9db6de2a22620 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:23:35 UTC) from {username='slave'} at 127.0.0.1:37430
I20260811 17:23:35.268888 6805 tablet_service.cc:1514] Processing DeleteTablet for tablet f2890b982efe4bc2a8a9db6de2a22620 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:23:35 UTC) from {username='slave'} at 127.0.0.1:37862
I20260811 17:23:35.269430 7206 tablet_replica.cc:330] T f2890b982efe4bc2a8a9db6de2a22620 P 0c2107b8a7f545b8868e3ec41044cba8: stopping tablet replica
I20260811 17:23:35.269594 7205 tablet_replica.cc:330] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab: stopping tablet replica
I20260811 17:23:35.269935 7205 raft_consensus.cc:2229] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:23:35.269935 7206 raft_consensus.cc:2229] T f2890b982efe4bc2a8a9db6de2a22620 P 0c2107b8a7f545b8868e3ec41044cba8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:23:35.270305 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 6485
I20260811 17:23:35.270344 7205 raft_consensus.cc:2258] T f2890b982efe4bc2a8a9db6de2a22620 P 914f15c163d34f17b7146896a5fcdcab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:23:35.270998 7204 tablet_replica.cc:330] T f2890b982efe4bc2a8a9db6de2a22620 P 929b85ca20a84de2bc8d35d9237be147: stopping tablet replica
I20260811 17:23:35.271286 7204 raft_consensus.cc:2229] T f2890b982efe4bc2a8a9db6de2a22620 P 929b85ca20a84de2bc8d35d9237be147 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:23:35.271569 7204 raft_consensus.cc:2258] T f2890b982efe4bc2a8a9db6de2a22620 P 929b85ca20a84de2bc8d35d9237be147 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:23:35.276388 7204 ts_tablet_manager.cc:1905] T f2890b982efe4bc2a8a9db6de2a22620 P 929b85ca20a84de2bc8d35d9237be147: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:23:35.279632 7206 raft_consensus.cc:2258] T f2890b982efe4bc2a8a9db6de2a22620 P 0c2107b8a7f545b8868e3ec41044cba8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:23:35.280400 7204 ts_tablet_manager.cc:1918] T f2890b982efe4bc2a8a9db6de2a22620 P 929b85ca20a84de2bc8d35d9237be147: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:23:35.280541 7204 log.cc:1196] T f2890b982efe4bc2a8a9db6de2a22620 P 929b85ca20a84de2bc8d35d9237be147: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-1/wal/wals/f2890b982efe4bc2a8a9db6de2a22620
I20260811 17:23:35.280967 7204 ts_tablet_manager.cc:1939] T f2890b982efe4bc2a8a9db6de2a22620 P 929b85ca20a84de2bc8d35d9237be147: Deleting consensus metadata
I20260811 17:23:35.281553 7206 ts_tablet_manager.cc:1905] T f2890b982efe4bc2a8a9db6de2a22620 P 0c2107b8a7f545b8868e3ec41044cba8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:23:35.282562 6373 catalog_manager.cc:4841] TS 929b85ca20a84de2bc8d35d9237be147 (127.9.157.130:37877): tablet f2890b982efe4bc2a8a9db6de2a22620 (table test-table [id=85113cc8c80743c5ad22f2a1470e44a7]) successfully deleted
I20260811 17:23:35.284085 7206 ts_tablet_manager.cc:1918] T f2890b982efe4bc2a8a9db6de2a22620 P 0c2107b8a7f545b8868e3ec41044cba8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:23:35.284214 7206 log.cc:1196] T f2890b982efe4bc2a8a9db6de2a22620 P 0c2107b8a7f545b8868e3ec41044cba8: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786468743878666-9846-0/minicluster-data/ts-2/wal/wals/f2890b982efe4bc2a8a9db6de2a22620
I20260811 17:23:35.284546 7206 ts_tablet_manager.cc:1939] T f2890b982efe4bc2a8a9db6de2a22620 P 0c2107b8a7f545b8868e3ec41044cba8: Deleting consensus metadata
I20260811 17:23:35.285822 6368 catalog_manager.cc:4841] TS 0c2107b8a7f545b8868e3ec41044cba8 (127.9.157.131:43403): tablet f2890b982efe4bc2a8a9db6de2a22620 (table test-table [id=85113cc8c80743c5ad22f2a1470e44a7]) successfully deleted
W20260811 17:23:35.287153 6371 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.157.129:44547 (error 108)
I20260811 17:23:35.287461 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 6703
I20260811 17:23:35.297086 7160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:35.296754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:44547 (local address 127.0.0.1:34884)
0811 17:23:35.296889 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:35.296987 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.129:44547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 17:23:35.297441 6371 catalog_manager.cc:4584] TS 914f15c163d34f17b7146896a5fcdcab (127.9.157.129:44547): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f2890b982efe4bc2a8a9db6de2a22620: Network error: Client connection negotiation failed: client connection to 127.9.157.129:44547: connect: Connection refused (error 111)
I20260811 17:23:35.303735 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 6924
I20260811 17:23:35.319269 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 6348
2026-08-11T17:23:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1424 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:23:35 dist-test-slave-mjjr krb5kdc[7213](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:23:35 dist-test-slave-mjjr krb5kdc[7213](info): set up 2 sockets
Aug 11 17:23:35 dist-test-slave-mjjr krb5kdc[7213](info): commencing operation
krb5kdc: starting...
W20260811 17:23:36.628319 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:23:36 dist-test-slave-mjjr krb5kdc[7213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:23:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:23:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:23:36.771023 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:37345
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:42457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:37345
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:23:36.895004 7236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:36.895341 7236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:36.895442 7236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:36.899600 7236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:23:36.899739 7236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:36.899780 7236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:36.899816 7236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:23:36.899855 7236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:23:36.905649 7236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:42457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:37345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:37345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:36.906908 7236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:36.908111 7236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:23:36.917217 7236 server_base.cc:1034] running on GCE node
W20260811 17:23:36.917187 7249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:36.917165 7246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:36.917169 7252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:36.918138 7236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:36.919368 7236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:36.920585 7236 hybrid_clock.cc:648] HybridClock initialized: now 1786469016920560 us; error 38 us; skew 500 ppm
Aug 11 17:23:36 dist-test-slave-mjjr krb5kdc[7213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469016, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:23:36.923636 7236 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:23:36.924721 7236 webserver.cc:460] Webserver started at http://127.9.157.190:32961/ using document root <none> and password file <none>
I20260811 17:23:36.925304 7236 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:36.925366 7236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:36.925550 7236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:36.927785 7236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "fc033b899107487e93e3228fe88243ed"
format_stamp: "Formatted at 2026-08-11 17:23:36 on dist-test-slave-mjjr"
server_key: "4c28a0160917a611481f409babffdb44"
server_key_iv: "edb06c47f95797d685a4345861c116a0"
server_key_version: "encryptionkey@0"
I20260811 17:23:36.928390 7236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "fc033b899107487e93e3228fe88243ed"
format_stamp: "Formatted at 2026-08-11 17:23:36 on dist-test-slave-mjjr"
server_key: "4c28a0160917a611481f409babffdb44"
server_key_iv: "edb06c47f95797d685a4345861c116a0"
server_key_version: "encryptionkey@0"
I20260811 17:23:36.932809 7236 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:23:36.936004 7261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:36.937438 7236 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 17:23:36.937569 7236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "fc033b899107487e93e3228fe88243ed"
format_stamp: "Formatted at 2026-08-11 17:23:36 on dist-test-slave-mjjr"
server_key: "4c28a0160917a611481f409babffdb44"
server_key_iv: "edb06c47f95797d685a4345861c116a0"
server_key_version: "encryptionkey@0"
I20260811 17:23:36.937685 7236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:36.959571 7236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:36.963213 7236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:36.963392 7236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:36.971426 7236 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:37345
I20260811 17:23:36.971462 7347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:37345 every 8 connection(s)
I20260811 17:23:36.972450 7236 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:23:36.975325 7348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:36.980067 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 7236
I20260811 17:23:36.980227 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:23:36.980559 9846 external_mini_cluster.cc:1428] Setting key 66028a3c233d8c3b62356ab181d5f16e
I20260811 17:23:36.981849 7348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc033b899107487e93e3228fe88243ed: Bootstrap starting.
I20260811 17:23:36.984539 7348 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc033b899107487e93e3228fe88243ed: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:36.985342 7348 log.cc:824] T 00000000000000000000000000000000 P fc033b899107487e93e3228fe88243ed: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:36.987365 7348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc033b899107487e93e3228fe88243ed: No bootstrap required, opened a new log
Aug 11 17:23:36 dist-test-slave-mjjr krb5kdc[7213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:23:36.991247 7348 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc033b899107487e93e3228fe88243ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc033b899107487e93e3228fe88243ed" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37345 } }
I20260811 17:23:36.991618 7348 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc033b899107487e93e3228fe88243ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:36.991712 7348 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc033b899107487e93e3228fe88243ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc033b899107487e93e3228fe88243ed, State: Initialized, Role: FOLLOWER
I20260811 17:23:36.992240 7348 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc033b899107487e93e3228fe88243ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc033b899107487e93e3228fe88243ed" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37345 } }
I20260811 17:23:36.992401 7348 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc033b899107487e93e3228fe88243ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:23:36.992497 7348 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc033b899107487e93e3228fe88243ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:23:36.992616 7348 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc033b899107487e93e3228fe88243ed [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:36.993821 7348 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc033b899107487e93e3228fe88243ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc033b899107487e93e3228fe88243ed" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37345 } }
I20260811 17:23:36.994239 7348 leader_election.cc:302] T 00000000000000000000000000000000 P fc033b899107487e93e3228fe88243ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc033b899107487e93e3228fe88243ed; no voters:
I20260811 17:23:36.994637 7348 leader_election.cc:288] T 00000000000000000000000000000000 P fc033b899107487e93e3228fe88243ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:23:36.994738 7357 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc033b899107487e93e3228fe88243ed [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:23:36.994916 7357 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc033b899107487e93e3228fe88243ed [term 1 LEADER]: Becoming Leader. State: Replica: fc033b899107487e93e3228fe88243ed, State: Running, Role: LEADER
I20260811 17:23:36.995291 7357 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc033b899107487e93e3228fe88243ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc033b899107487e93e3228fe88243ed" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37345 } }
I20260811 17:23:36.995800 7354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:36.982187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48572 (local address 127.9.157.190:37345)
0811 17:23:36.982679 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:36.982697 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:36.982737 (+ 40us) server_negotiation.cc:407] Connection header received
0811 17:23:36.983448 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:36.983466 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:36.983905 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:36.984262 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:36.985138 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:36.986196 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:36.987083 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:36.987627 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:36.987863 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:36.990436 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:36.990443 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:36.990458 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:23:36.990487 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:36.992356 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:36.992982 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:36.992988 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:36.992992 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:36.993106 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:36.993413 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:36.993415 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:36.993416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:36.993829 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:36.994019 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:36.994322 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 17:23:36.994720 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":151,"threads_started":1}
I20260811 17:23:36.996047 7348 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc033b899107487e93e3228fe88243ed [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:23:36.997133 7361 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc033b899107487e93e3228fe88243ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc033b899107487e93e3228fe88243ed. Latest consensus state: current_term: 1 leader_uuid: "fc033b899107487e93e3228fe88243ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc033b899107487e93e3228fe88243ed" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37345 } } }
I20260811 17:23:36.997256 7361 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc033b899107487e93e3228fe88243ed [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:36.997550 7360 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc033b899107487e93e3228fe88243ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc033b899107487e93e3228fe88243ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc033b899107487e93e3228fe88243ed" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37345 } } }
I20260811 17:23:36.997642 7360 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc033b899107487e93e3228fe88243ed [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:36.998108 7370 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:23:37.001019 7370 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:23:37.007089 7370 catalog_manager.cc:1347] Generated new cluster ID: 44c98280858c4ff48bc19f5e50fd5a9c
I20260811 17:23:37.007174 7370 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:23:37.028527 7370 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:23:37.029989 7370 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:23:37.048635 7370 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc033b899107487e93e3228fe88243ed: Generated new TSK 0
I20260811 17:23:37.049464 7370 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.157.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
Entry for principal kudu/127.9.157.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab.
I20260811 17:23:37.082688 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:0
--local_ip_for_outbound_sockets=127.9.157.129
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:37345
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:42457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:23:37.204389 7386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:37.204680 7386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:37.204727 7386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:37.209077 7386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:37.209151 7386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:37.209213 7386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:23:37.214814 7386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:42457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.129.keytab
--principal=kudu/127.9.157.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:37345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:37.216094 7386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:37.217197 7386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:37.227787 7401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:37.227794 7398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:37.227845 7397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:37.228186 7386 server_base.cc:1034] running on GCE node
I20260811 17:23:37.228678 7386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:37.229396 7386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:37.230577 7386 hybrid_clock.cc:648] HybridClock initialized: now 1786469017230545 us; error 44 us; skew 500 ppm
Aug 11 17:23:37 dist-test-slave-mjjr krb5kdc[7213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469017, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:23:37.233582 7386 init.cc:377] Logged in from keytab as kudu/127.9.157.129@KRBTEST.COM (short username kudu)
I20260811 17:23:37.234597 7386 webserver.cc:460] Webserver started at http://127.9.157.129:45909/ using document root <none> and password file <none>
I20260811 17:23:37.235164 7386 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:37.235227 7386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:37.235405 7386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:37.237586 7386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-0/data/instance:
uuid: "434f9ce8071943428076a659fd2cdfa2"
format_stamp: "Formatted at 2026-08-11 17:23:37 on dist-test-slave-mjjr"
server_key: "983145497f7f13441835234eb2b7f8b4"
server_key_iv: "17e50be15e24183bc4f3a02c595a3b17"
server_key_version: "encryptionkey@0"
I20260811 17:23:37.238144 7386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance:
uuid: "434f9ce8071943428076a659fd2cdfa2"
format_stamp: "Formatted at 2026-08-11 17:23:37 on dist-test-slave-mjjr"
server_key: "983145497f7f13441835234eb2b7f8b4"
server_key_iv: "17e50be15e24183bc4f3a02c595a3b17"
server_key_version: "encryptionkey@0"
I20260811 17:23:37.242182 7386 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:23:37.244799 7410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:37.246039 7386 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 17:23:37.246178 7386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "434f9ce8071943428076a659fd2cdfa2"
format_stamp: "Formatted at 2026-08-11 17:23:37 on dist-test-slave-mjjr"
server_key: "983145497f7f13441835234eb2b7f8b4"
server_key_iv: "17e50be15e24183bc4f3a02c595a3b17"
server_key_version: "encryptionkey@0"
I20260811 17:23:37.246289 7386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:37.276279 7386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:37.279848 7386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:37.280114 7386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:37.280932 7386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:23:37.281896 7386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:23:37.281956 7386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:37.282001 7386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:23:37.282016 7386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:37.292343 7386 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:40345
I20260811 17:23:37.292366 7609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:40345 every 8 connection(s)
I20260811 17:23:37.293382 7386 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:23:37.302148 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 7386
I20260811 17:23:37.302318 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance
I20260811 17:23:37.302755 9846 external_mini_cluster.cc:1428] Setting key b21b6f635555396e321f0964989dd29e
Aug 11 17:23:37 dist-test-slave-mjjr krb5kdc[7213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469017, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.129@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:23:37.309516 7354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:37.295384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:44433 (local address 127.9.157.190:37345)
0811 17:23:37.295576 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:37.295582 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:37.296552 (+ 970us) server_negotiation.cc:407] Connection header received
0811 17:23:37.298043 (+ 1491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:37.298046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:37.298102 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:37.298188 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:37.299429 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:37.300158 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:37.301307 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:37.301576 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:37.301672 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:37.304908 (+ 3236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:37.304912 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:37.304914 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:23:37.304941 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:37.306633 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:37.307369 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:37.307375 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:37.307380 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:37.307501 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:37.307999 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:37.308004 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:37.308008 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:37.308291 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:37.308412 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:37.309075 (+ 663us) server_negotiation.cc:299] Negotiation successful
0811 17:23:37.309242 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 17:23:37.310444 7612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:37.295888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:37345 (local address 127.9.157.129:44433)
0811 17:23:37.296381 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:37.296431 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:37.297844 (+ 1413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:37.298373 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:37.298379 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:37.298745 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:23:37.299253 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:37.299267 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:37.300286 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:37.300288 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:37.301179 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:37.301188 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:37.301725 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:37.301728 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:37.301941 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:37.302690 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:23:37.302714 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:23:37.304685 (+ 1971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:23:37.306768 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:23:37.306778 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:23:37.306801 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:23:37.307254 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:23:37.307629 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:23:37.307632 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:23:37.307634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:23:37.307891 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:23:37.308442 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:23:37.308449 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:23:37.308774 (+ 325us) client_negotiation.cc:769] Sending connection context
0811 17:23:37.308964 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 17:23:37.309282 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":134,"threads_started":1}
I20260811 17:23:37.311995 7610 heartbeater.cc:347] Connected to a master server at 127.9.157.190:37345
I20260811 17:23:37.312275 7610 heartbeater.cc:464] Registering TS with master...
I20260811 17:23:37.312851 7610 heartbeater.cc:511] Master 127.9.157.190:37345 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.157.130@KRBTEST.COM; defaulting to no policy
I20260811 17:23:37.314569 7283 ts_manager.cc:194] Registered new tserver with Master: 434f9ce8071943428076a659fd2cdfa2 (127.9.157.129:40345)
I20260811 17:23:37.315896 7283 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.129@KRBTEST.COM'} at 127.9.157.129:44433
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
Entry for principal kudu/127.9.157.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab.
I20260811 17:23:37.335609 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:0
--local_ip_for_outbound_sockets=127.9.157.130
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:37345
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:42457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:23:37.456009 7619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:37.456331 7619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:37.456422 7619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:37.460649 7619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:37.460731 7619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:37.460819 7619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:23:37.466393 7619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:42457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.130.keytab
--principal=kudu/127.9.157.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:37345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:37.467734 7619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:37.468899 7619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:37.479442 7632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:37.479419 7631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:37.479420 7634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:37.479880 7619 server_base.cc:1034] running on GCE node
I20260811 17:23:37.480330 7619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:37.481057 7619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:37.482251 7619 hybrid_clock.cc:648] HybridClock initialized: now 1786469017482222 us; error 39 us; skew 500 ppm
Aug 11 17:23:37 dist-test-slave-mjjr krb5kdc[7213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469017, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:23:37.485244 7619 init.cc:377] Logged in from keytab as kudu/127.9.157.130@KRBTEST.COM (short username kudu)
I20260811 17:23:37.486241 7619 webserver.cc:460] Webserver started at http://127.9.157.130:38923/ using document root <none> and password file <none>
I20260811 17:23:37.486873 7619 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:37.486974 7619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:37.487236 7619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:37.489499 7619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-1/data/instance:
uuid: "411abab0828a4f97a2365026a5e0c6d6"
format_stamp: "Formatted at 2026-08-11 17:23:37 on dist-test-slave-mjjr"
server_key: "11994fcd62ef3e1be3ee32b8c95b3822"
server_key_iv: "ddded656e40720120e2ccd1d0e148530"
server_key_version: "encryptionkey@0"
I20260811 17:23:37.490120 7619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance:
uuid: "411abab0828a4f97a2365026a5e0c6d6"
format_stamp: "Formatted at 2026-08-11 17:23:37 on dist-test-slave-mjjr"
server_key: "11994fcd62ef3e1be3ee32b8c95b3822"
server_key_iv: "ddded656e40720120e2ccd1d0e148530"
server_key_version: "encryptionkey@0"
I20260811 17:23:37.494294 7619 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 17:23:37.497092 7644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:37.498632 7619 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:23:37.498778 7619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "411abab0828a4f97a2365026a5e0c6d6"
format_stamp: "Formatted at 2026-08-11 17:23:37 on dist-test-slave-mjjr"
server_key: "11994fcd62ef3e1be3ee32b8c95b3822"
server_key_iv: "ddded656e40720120e2ccd1d0e148530"
server_key_version: "encryptionkey@0"
I20260811 17:23:37.498907 7619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:37.519049 7619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:37.522343 7619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:37.522583 7619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:37.523238 7619 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:23:37.524286 7619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:23:37.524360 7619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:37.524435 7619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:23:37.524472 7619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:37.534116 7619 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:44541
I20260811 17:23:37.534168 7836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:44541 every 8 connection(s)
I20260811 17:23:37.535156 7619 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:23:37.544340 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 7619
I20260811 17:23:37.544504 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance
I20260811 17:23:37.544812 9846 external_mini_cluster.cc:1428] Setting key 3bb365e748c51431c9c41892e3711208
Aug 11 17:23:37 dist-test-slave-mjjr krb5kdc[7213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469017, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.130@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:23:37.551122 7354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:37.537278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:59447 (local address 127.9.157.190:37345)
0811 17:23:37.537470 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:37.537475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:37.538403 (+ 928us) server_negotiation.cc:407] Connection header received
0811 17:23:37.539936 (+ 1533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:37.539939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:37.539995 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:37.540066 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:37.541383 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:37.542021 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:37.543373 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:37.543600 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:37.543795 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:37.546800 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:37.546804 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:37.546806 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:23:37.546831 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:37.548505 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:37.549138 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:37.549144 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:37.549146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:37.549247 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:37.549656 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:37.549662 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:37.549664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:37.549968 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:37.550072 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:37.550739 (+ 667us) server_negotiation.cc:299] Negotiation successful
0811 17:23:37.550898 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 17:23:37.552134 7845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:37.537683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:37345 (local address 127.9.157.130:59447)
0811 17:23:37.538214 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:37.538262 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:37.539601 (+ 1339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:37.540283 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:37.540292 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:37.540687 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:23:37.541230 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:37.541245 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:37.542161 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:37.542164 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:37.543235 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:37.543246 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:37.543863 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:37.543866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:37.544042 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:37.544682 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:23:37.544699 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:23:37.546586 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:23:37.548630 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:23:37.548641 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:23:37.548664 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:23:37.549030 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:23:37.549348 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:23:37.549351 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:23:37.549352 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:23:37.549556 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:23:37.550087 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:23:37.550092 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:23:37.550415 (+ 323us) client_negotiation.cc:769] Sending connection context
0811 17:23:37.550691 (+ 276us) client_negotiation.cc:240] Negotiation successful
0811 17:23:37.551092 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":122,"threads_started":1}
I20260811 17:23:37.553483 7839 heartbeater.cc:347] Connected to a master server at 127.9.157.190:37345
I20260811 17:23:37.553772 7839 heartbeater.cc:464] Registering TS with master...
I20260811 17:23:37.554364 7839 heartbeater.cc:511] Master 127.9.157.190:37345 requested a full tablet report, sending...
I20260811 17:23:37.555625 7283 ts_manager.cc:194] Registered new tserver with Master: 411abab0828a4f97a2365026a5e0c6d6 (127.9.157.130:44541)
I20260811 17:23:37.556545 7283 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.130@KRBTEST.COM'} at 127.9.157.130:59447
WARNING: no policy specified for kudu/127.9.157.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
Entry for principal kudu/127.9.157.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab.
I20260811 17:23:37.578689 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:0
--local_ip_for_outbound_sockets=127.9.157.131
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:37345
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:42457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:23:37.700558 7849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:37.700875 7849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:37.700922 7849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:37.705320 7849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:37.705415 7849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:37.705478 7849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:23:37.711380 7849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:42457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.131.keytab
--principal=kudu/127.9.157.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:37345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:37.712672 7849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:37.713757 7849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:37.724323 7854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:37.724416 7857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:37.724377 7861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:37.724392 7849 server_base.cc:1034] running on GCE node
I20260811 17:23:37.725121 7849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:37.725859 7849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:37.727066 7849 hybrid_clock.cc:648] HybridClock initialized: now 1786469017727050 us; error 26 us; skew 500 ppm
Aug 11 17:23:37 dist-test-slave-mjjr krb5kdc[7213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469017, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:23:37.730202 7849 init.cc:377] Logged in from keytab as kudu/127.9.157.131@KRBTEST.COM (short username kudu)
I20260811 17:23:37.731174 7849 webserver.cc:460] Webserver started at http://127.9.157.131:40995/ using document root <none> and password file <none>
I20260811 17:23:37.731868 7849 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:37.731977 7849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:37.732229 7849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:37.734522 7849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-2/data/instance:
uuid: "c5a46e80c87e4bd196a970caa59611bc"
format_stamp: "Formatted at 2026-08-11 17:23:37 on dist-test-slave-mjjr"
server_key: "7a7614d25b7c3b0afa067db35103137d"
server_key_iv: "88f02156b3acbc4444ccd7b2d9b99341"
server_key_version: "encryptionkey@0"
I20260811 17:23:37.735142 7849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance:
uuid: "c5a46e80c87e4bd196a970caa59611bc"
format_stamp: "Formatted at 2026-08-11 17:23:37 on dist-test-slave-mjjr"
server_key: "7a7614d25b7c3b0afa067db35103137d"
server_key_iv: "88f02156b3acbc4444ccd7b2d9b99341"
server_key_version: "encryptionkey@0"
I20260811 17:23:37.739462 7849 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:23:37.742144 7873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:37.743458 7849 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 17:23:37.743604 7849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "c5a46e80c87e4bd196a970caa59611bc"
format_stamp: "Formatted at 2026-08-11 17:23:37 on dist-test-slave-mjjr"
server_key: "7a7614d25b7c3b0afa067db35103137d"
server_key_iv: "88f02156b3acbc4444ccd7b2d9b99341"
server_key_version: "encryptionkey@0"
I20260811 17:23:37.743803 7849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:37.760654 7849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:37.764209 7849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:37.764448 7849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:37.765151 7849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:23:37.766103 7849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:23:37.766291 7849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:37.766434 7849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:23:37.766499 7849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:37.777750 7849 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:39135
I20260811 17:23:37.777782 8072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:39135 every 8 connection(s)
I20260811 17:23:37.778802 7849 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:23:37.787621 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 7849
I20260811 17:23:37.787824 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance
I20260811 17:23:37.788177 9846 external_mini_cluster.cc:1428] Setting key 505c3ef871561120d02c57997b293957
Aug 11 17:23:37 dist-test-slave-mjjr krb5kdc[7213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469017, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.131@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:23:37.794536 7354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:37.781031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:42405 (local address 127.9.157.190:37345)
0811 17:23:37.781219 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:37.781223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:37.782225 (+ 1002us) server_negotiation.cc:407] Connection header received
0811 17:23:37.783955 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:37.783959 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:37.784031 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:37.784145 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:37.785490 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:37.786065 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:37.787396 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:37.787830 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:37.787954 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:37.790960 (+ 3006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:37.790963 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:37.790965 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:23:37.790991 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:37.792439 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:37.792968 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:37.792971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:37.792972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:37.793045 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:37.793364 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:37.793366 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:37.793367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:37.793558 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:37.793634 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:37.794201 (+ 567us) server_negotiation.cc:299] Negotiation successful
0811 17:23:37.794316 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 17:23:37.795356 8075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:37.781438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:37345 (local address 127.9.157.131:42405)
0811 17:23:37.782033 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:37.782086 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:37.783607 (+ 1521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:37.784356 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:37.784364 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:37.784773 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:23:37.785316 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:37.785331 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:37.786250 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:37.786253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:37.787244 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:37.787256 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:37.788031 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:37.788034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:37.788201 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:37.788973 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:23:37.788995 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:23:37.790792 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:23:37.792557 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:23:37.792563 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:23:37.792578 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:23:37.792878 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:23:37.793128 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:23:37.793130 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:23:37.793131 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:23:37.793274 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:23:37.793650 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:23:37.793655 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:23:37.793922 (+ 267us) client_negotiation.cc:769] Sending connection context
0811 17:23:37.794099 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 17:23:37.794395 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":138,"threads_started":1}
I20260811 17:23:37.796664 8073 heartbeater.cc:347] Connected to a master server at 127.9.157.190:37345
I20260811 17:23:37.796943 8073 heartbeater.cc:464] Registering TS with master...
I20260811 17:23:37.797602 8073 heartbeater.cc:511] Master 127.9.157.190:37345 requested a full tablet report, sending...
I20260811 17:23:37.798736 7283 ts_manager.cc:194] Registered new tserver with Master: c5a46e80c87e4bd196a970caa59611bc (127.9.157.131:39135)
I20260811 17:23:37.799327 7283 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.157.131@KRBTEST.COM'} at 127.9.157.131:42405
I20260811 17:23:37.802584 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:23:37.813251 7354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:37.804806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48580 (local address 127.9.157.190:37345)
0811 17:23:37.804938 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:37.804941 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:37.805124 (+ 183us) server_negotiation.cc:407] Connection header received
0811 17:23:37.805246 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:37.805248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:37.805292 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:37.805352 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:37.806148 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:37.806685 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:37.807718 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:37.808017 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:37.808119 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:37.809702 (+ 1583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:37.809706 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:37.809708 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:23:37.809764 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:37.811224 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:37.811770 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:37.811774 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:37.811776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:37.811880 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:37.812219 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:37.812223 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:37.812224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:37.812524 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:37.812660 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:37.812986 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 17:23:37.813108 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 17:23:37.818216 7283 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48580:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:23:37.821888 7283 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:23:37.839027 8088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:37.832176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:40345 (local address 127.0.0.1:52504)
0811 17:23:37.832975 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:37.833036 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:37.833205 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:37.834052 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:37.834059 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:37.834090 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:37.834301 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:37.834310 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:37.835542 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:37.835547 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:37.837165 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:37.837177 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:37.838320 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:37.838324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:37.838450 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:37.838507 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 17:23:37.838624 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 17:23:37.838809 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"mutex_wait_us":86,"spinlock_wait_cycles":7424,"thread_start_us":121,"threads_started":1}
I20260811 17:23:37.839075 8096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:37.833516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44568 (local address 127.9.157.131:39135)
0811 17:23:37.834388 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:37.834402 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:37.834428 (+ 26us) server_negotiation.cc:407] Connection header received
0811 17:23:37.834524 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:37.834530 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:37.834718 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:37.834888 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:37.835147 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:37.836295 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:37.837752 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:37.838402 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:37.838517 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:37.838660 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:37.838815 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 17:23:37.838929 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":762,"thread_start_us":162,"threads_started":1}
I20260811 17:23:37.839571 8089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:37.832363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52504 (local address 127.9.157.129:40345)
0811 17:23:37.833521 (+ 1158us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:37.833531 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:37.833558 (+ 27us) server_negotiation.cc:407] Connection header received
0811 17:23:37.833647 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:37.833653 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:37.833881 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:37.834047 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:37.834453 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:37.835432 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:37.837314 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:37.838173 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:37.838281 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:37.838410 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:37.838929 (+ 519us) server_negotiation.cc:299] Negotiation successful
0811 17:23:37.839027 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1044,"thread_start_us":151,"threads_started":1}
I20260811 17:23:37.839686 8094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:37.833018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:39135 (local address 127.0.0.1:44568)
0811 17:23:37.833982 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:37.834000 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:37.834105 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:37.834886 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:37.834889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:37.834909 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:37.835010 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:37.835015 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:37.836459 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:37.836463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:37.837612 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:37.837623 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:37.838544 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:37.838548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:37.838655 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:37.838681 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 17:23:37.838765 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 17:23:37.838840 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":887,"spinlock_wait_cycles":3200,"thread_start_us":113,"threads_started":1}
I20260811 17:23:37.840054 8090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:37.832451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:44541 (local address 127.0.0.1:52334)
0811 17:23:37.833013 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:37.833059 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:37.833203 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:37.834100 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:37.834103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:37.834120 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:37.834315 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:37.834321 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:37.835829 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:37.835833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:37.837165 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:37.837177 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:37.839601 (+ 2424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:37.839605 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:37.839738 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:37.839770 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 17:23:37.839841 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 17:23:37.839922 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":98,"spinlock_wait_cycles":13184,"thread_start_us":131,"threads_started":1}
I20260811 17:23:37.840703 8092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:37.832618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52334 (local address 127.9.157.130:44541)
0811 17:23:37.833534 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:37.833548 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:37.833576 (+ 28us) server_negotiation.cc:407] Connection header received
0811 17:23:37.833667 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:37.833676 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:37.833914 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:37.834275 (+ 361us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:37.834467 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:37.835642 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:37.838506 (+ 2864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:37.839460 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:37.840239 (+ 779us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:37.840392 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:37.840446 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 17:23:37.840537 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":800,"thread_start_us":95,"threads_started":1}
I20260811 17:23:37.842926 7485 tablet_service.cc:1467] Processing CreateTablet for tablet 37712c2f2b834bdcaf5f7d1d027e9d24 (DEFAULT_TABLE table=test-table [id=4425fdf618e84a32a98d445b573928e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:23:37.842931 7957 tablet_service.cc:1467] Processing CreateTablet for tablet 37712c2f2b834bdcaf5f7d1d027e9d24 (DEFAULT_TABLE table=test-table [id=4425fdf618e84a32a98d445b573928e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:23:37.843288 7717 tablet_service.cc:1467] Processing CreateTablet for tablet 37712c2f2b834bdcaf5f7d1d027e9d24 (DEFAULT_TABLE table=test-table [id=4425fdf618e84a32a98d445b573928e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:23:37.844151 7485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37712c2f2b834bdcaf5f7d1d027e9d24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:37.844162 7957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37712c2f2b834bdcaf5f7d1d027e9d24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:37.844233 7717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37712c2f2b834bdcaf5f7d1d027e9d24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:37.851383 8097 tablet_bootstrap.cc:492] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 434f9ce8071943428076a659fd2cdfa2: Bootstrap starting.
I20260811 17:23:37.852232 8098 tablet_bootstrap.cc:492] T 37712c2f2b834bdcaf5f7d1d027e9d24 P c5a46e80c87e4bd196a970caa59611bc: Bootstrap starting.
I20260811 17:23:37.854211 8097 tablet_bootstrap.cc:654] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 434f9ce8071943428076a659fd2cdfa2: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:37.854211 8098 tablet_bootstrap.cc:654] T 37712c2f2b834bdcaf5f7d1d027e9d24 P c5a46e80c87e4bd196a970caa59611bc: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:37.854764 8099 tablet_bootstrap.cc:492] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6: Bootstrap starting.
I20260811 17:23:37.855464 8097 log.cc:824] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 434f9ce8071943428076a659fd2cdfa2: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:37.855623 8098 log.cc:824] T 37712c2f2b834bdcaf5f7d1d027e9d24 P c5a46e80c87e4bd196a970caa59611bc: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:37.857807 8099 tablet_bootstrap.cc:654] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:37.858105 8097 tablet_bootstrap.cc:492] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 434f9ce8071943428076a659fd2cdfa2: No bootstrap required, opened a new log
I20260811 17:23:37.858331 8097 ts_tablet_manager.cc:1397] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 434f9ce8071943428076a659fd2cdfa2: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 17:23:37.858714 8099 log.cc:824] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:37.860611 8099 tablet_bootstrap.cc:492] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6: No bootstrap required, opened a new log
I20260811 17:23:37.860723 8098 tablet_bootstrap.cc:492] T 37712c2f2b834bdcaf5f7d1d027e9d24 P c5a46e80c87e4bd196a970caa59611bc: No bootstrap required, opened a new log
I20260811 17:23:37.860854 8099 ts_tablet_manager.cc:1397] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 17:23:37.860929 8098 ts_tablet_manager.cc:1397] T 37712c2f2b834bdcaf5f7d1d027e9d24 P c5a46e80c87e4bd196a970caa59611bc: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.004s
I20260811 17:23:37.863286 8097 raft_consensus.cc:348] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 434f9ce8071943428076a659fd2cdfa2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "434f9ce8071943428076a659fd2cdfa2" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40345 } } peers { permanent_uuid: "c5a46e80c87e4bd196a970caa59611bc" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39135 } } peers { permanent_uuid: "411abab0828a4f97a2365026a5e0c6d6" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 44541 } }
I20260811 17:23:37.863600 8097 raft_consensus.cc:374] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 434f9ce8071943428076a659fd2cdfa2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:37.863687 8097 raft_consensus.cc:729] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 434f9ce8071943428076a659fd2cdfa2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 434f9ce8071943428076a659fd2cdfa2, State: Initialized, Role: FOLLOWER
I20260811 17:23:37.864225 8097 consensus_queue.cc:260] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 434f9ce8071943428076a659fd2cdfa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "434f9ce8071943428076a659fd2cdfa2" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40345 } } peers { permanent_uuid: "c5a46e80c87e4bd196a970caa59611bc" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39135 } } peers { permanent_uuid: "411abab0828a4f97a2365026a5e0c6d6" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 44541 } }
I20260811 17:23:37.864534 8099 raft_consensus.cc:348] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "434f9ce8071943428076a659fd2cdfa2" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40345 } } peers { permanent_uuid: "c5a46e80c87e4bd196a970caa59611bc" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39135 } } peers { permanent_uuid: "411abab0828a4f97a2365026a5e0c6d6" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 44541 } }
I20260811 17:23:37.864835 8099 raft_consensus.cc:374] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:37.864893 8099 raft_consensus.cc:729] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 411abab0828a4f97a2365026a5e0c6d6, State: Initialized, Role: FOLLOWER
I20260811 17:23:37.864861 8098 raft_consensus.cc:348] T 37712c2f2b834bdcaf5f7d1d027e9d24 P c5a46e80c87e4bd196a970caa59611bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "434f9ce8071943428076a659fd2cdfa2" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40345 } } peers { permanent_uuid: "c5a46e80c87e4bd196a970caa59611bc" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39135 } } peers { permanent_uuid: "411abab0828a4f97a2365026a5e0c6d6" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 44541 } }
I20260811 17:23:37.865110 8098 raft_consensus.cc:374] T 37712c2f2b834bdcaf5f7d1d027e9d24 P c5a46e80c87e4bd196a970caa59611bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:37.865159 8098 raft_consensus.cc:729] T 37712c2f2b834bdcaf5f7d1d027e9d24 P c5a46e80c87e4bd196a970caa59611bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5a46e80c87e4bd196a970caa59611bc, State: Initialized, Role: FOLLOWER
I20260811 17:23:37.865420 8097 ts_tablet_manager.cc:1428] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 434f9ce8071943428076a659fd2cdfa2: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 17:23:37.865414 8099 consensus_queue.cc:260] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "434f9ce8071943428076a659fd2cdfa2" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40345 } } peers { permanent_uuid: "c5a46e80c87e4bd196a970caa59611bc" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39135 } } peers { permanent_uuid: "411abab0828a4f97a2365026a5e0c6d6" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 44541 } }
I20260811 17:23:37.865613 8098 consensus_queue.cc:260] T 37712c2f2b834bdcaf5f7d1d027e9d24 P c5a46e80c87e4bd196a970caa59611bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "434f9ce8071943428076a659fd2cdfa2" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40345 } } peers { permanent_uuid: "c5a46e80c87e4bd196a970caa59611bc" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39135 } } peers { permanent_uuid: "411abab0828a4f97a2365026a5e0c6d6" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 44541 } }
I20260811 17:23:37.865825 7610 heartbeater.cc:503] Master 127.9.157.190:37345 was elected leader, sending a full tablet report...
I20260811 17:23:37.866567 8099 ts_tablet_manager.cc:1428] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 17:23:37.866588 8098 ts_tablet_manager.cc:1428] T 37712c2f2b834bdcaf5f7d1d027e9d24 P c5a46e80c87e4bd196a970caa59611bc: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 17:23:37.866947 7839 heartbeater.cc:503] Master 127.9.157.190:37345 was elected leader, sending a full tablet report...
I20260811 17:23:37.866922 8073 heartbeater.cc:503] Master 127.9.157.190:37345 was elected leader, sending a full tablet report...
W20260811 17:23:38.030085 8074 tablet.cc:2369] T 37712c2f2b834bdcaf5f7d1d027e9d24 P c5a46e80c87e4bd196a970caa59611bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:23:38.036521 7843 tablet.cc:2369] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:23:38.044853 7611 tablet.cc:2369] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 434f9ce8071943428076a659fd2cdfa2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:23:38.226791 8109 raft_consensus.cc:482] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:23:38.227072 8109 raft_consensus.cc:504] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "434f9ce8071943428076a659fd2cdfa2" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40345 } } peers { permanent_uuid: "c5a46e80c87e4bd196a970caa59611bc" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39135 } } peers { permanent_uuid: "411abab0828a4f97a2365026a5e0c6d6" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 44541 } }
I20260811 17:23:38.228416 8109 leader_election.cc:288] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 434f9ce8071943428076a659fd2cdfa2 (127.9.157.129:40345), c5a46e80c87e4bd196a970caa59611bc (127.9.157.131:39135)
I20260811 17:23:38.234460 8096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:38.229451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:37033 (local address 127.9.157.131:39135)
0811 17:23:38.229619 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:38.229627 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:38.229723 (+ 96us) server_negotiation.cc:407] Connection header received
0811 17:23:38.229814 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:38.229816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:38.229855 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:38.229923 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:38.230204 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:38.230931 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:38.233062 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:38.233727 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:38.233873 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:38.233934 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:38.234222 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 17:23:38.234310 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 17:23:38.234460 8089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:38.229057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:50793 (local address 127.9.157.129:40345)
0811 17:23:38.229257 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:38.229262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:38.229282 (+ 20us) server_negotiation.cc:407] Connection header received
0811 17:23:38.229352 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:38.229355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:38.229428 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:38.229528 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:38.230032 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:38.230868 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:38.233062 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:38.233719 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:38.233882 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:38.233934 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:38.234222 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 17:23:38.234310 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 17:23:38.234555 8117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:38.229255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:39135 (local address 127.9.157.130:37033)
0811 17:23:38.229614 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:38.229628 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:38.229733 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:38.229955 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:38.229960 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:38.229982 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:38.230097 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:38.230101 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:38.231061 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:38.231065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:38.232857 (+ 1792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:38.232871 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:38.233872 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:38.233876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:38.234046 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:38.234088 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 17:23:38.234173 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 17:23:38.234335 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":294,"spinlock_wait_cycles":2944,"thread_start_us":106,"threads_started":1}
I20260811 17:23:38.234562 8114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:38.228825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:40345 (local address 127.9.157.130:50793)
0811 17:23:38.229128 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:38.229147 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:38.229259 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:38.229690 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:38.229695 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:38.229717 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:38.229873 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:38.229880 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:38.231061 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:38.231065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:38.232857 (+ 1792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:38.232871 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:38.233872 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:38.233876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:38.234039 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:38.234096 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 17:23:38.234173 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 17:23:38.234338 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":307,"thread_start_us":114,"threads_started":1}
I20260811 17:23:38.235324 7529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37712c2f2b834bdcaf5f7d1d027e9d24" candidate_uuid: "411abab0828a4f97a2365026a5e0c6d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434f9ce8071943428076a659fd2cdfa2" is_pre_election: true
I20260811 17:23:38.235447 7997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37712c2f2b834bdcaf5f7d1d027e9d24" candidate_uuid: "411abab0828a4f97a2365026a5e0c6d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5a46e80c87e4bd196a970caa59611bc" is_pre_election: true
I20260811 17:23:38.235785 7529 raft_consensus.cc:2454] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 434f9ce8071943428076a659fd2cdfa2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 411abab0828a4f97a2365026a5e0c6d6 in term 0.
I20260811 17:23:38.235802 7997 raft_consensus.cc:2454] T 37712c2f2b834bdcaf5f7d1d027e9d24 P c5a46e80c87e4bd196a970caa59611bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 411abab0828a4f97a2365026a5e0c6d6 in term 0.
I20260811 17:23:38.236438 7645 leader_election.cc:302] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 411abab0828a4f97a2365026a5e0c6d6, c5a46e80c87e4bd196a970caa59611bc; no voters:
I20260811 17:23:38.236779 8109 raft_consensus.cc:2781] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:23:38.236876 8109 raft_consensus.cc:482] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:23:38.236933 8109 raft_consensus.cc:3037] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:38.238077 8109 raft_consensus.cc:504] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "434f9ce8071943428076a659fd2cdfa2" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40345 } } peers { permanent_uuid: "c5a46e80c87e4bd196a970caa59611bc" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39135 } } peers { permanent_uuid: "411abab0828a4f97a2365026a5e0c6d6" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 44541 } }
I20260811 17:23:38.238466 8109 leader_election.cc:288] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6 [CANDIDATE]: Term 1 election: Requested vote from peers 434f9ce8071943428076a659fd2cdfa2 (127.9.157.129:40345), c5a46e80c87e4bd196a970caa59611bc (127.9.157.131:39135)
I20260811 17:23:38.238983 7529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37712c2f2b834bdcaf5f7d1d027e9d24" candidate_uuid: "411abab0828a4f97a2365026a5e0c6d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "434f9ce8071943428076a659fd2cdfa2"
I20260811 17:23:38.239004 7997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37712c2f2b834bdcaf5f7d1d027e9d24" candidate_uuid: "411abab0828a4f97a2365026a5e0c6d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5a46e80c87e4bd196a970caa59611bc"
I20260811 17:23:38.239135 7529 raft_consensus.cc:3037] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 434f9ce8071943428076a659fd2cdfa2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:38.239149 7997 raft_consensus.cc:3037] T 37712c2f2b834bdcaf5f7d1d027e9d24 P c5a46e80c87e4bd196a970caa59611bc [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:38.240360 7529 raft_consensus.cc:2454] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 434f9ce8071943428076a659fd2cdfa2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 411abab0828a4f97a2365026a5e0c6d6 in term 1.
I20260811 17:23:38.240361 7997 raft_consensus.cc:2454] T 37712c2f2b834bdcaf5f7d1d027e9d24 P c5a46e80c87e4bd196a970caa59611bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 411abab0828a4f97a2365026a5e0c6d6 in term 1.
I20260811 17:23:38.240721 7645 leader_election.cc:302] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 411abab0828a4f97a2365026a5e0c6d6, c5a46e80c87e4bd196a970caa59611bc; no voters:
I20260811 17:23:38.241037 8109 raft_consensus.cc:2781] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:23:38.241346 8109 raft_consensus.cc:686] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6 [term 1 LEADER]: Becoming Leader. State: Replica: 411abab0828a4f97a2365026a5e0c6d6, State: Running, Role: LEADER
I20260811 17:23:38.241791 8109 consensus_queue.cc:237] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "434f9ce8071943428076a659fd2cdfa2" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40345 } } peers { permanent_uuid: "c5a46e80c87e4bd196a970caa59611bc" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39135 } } peers { permanent_uuid: "411abab0828a4f97a2365026a5e0c6d6" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 44541 } }
I20260811 17:23:38.245378 7283 catalog_manager.cc:5495] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 411abab0828a4f97a2365026a5e0c6d6 (127.9.157.130). New cstate: current_term: 1 leader_uuid: "411abab0828a4f97a2365026a5e0c6d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "434f9ce8071943428076a659fd2cdfa2" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5a46e80c87e4bd196a970caa59611bc" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "411abab0828a4f97a2365026a5e0c6d6" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 44541 } health_report { overall_health: HEALTHY } } }
I20260811 17:23:38.294592 8092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:38.290711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52342 (local address 127.9.157.130:44541)
0811 17:23:38.290876 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:38.290883 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:38.290903 (+ 20us) server_negotiation.cc:407] Connection header received
0811 17:23:38.291001 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:38.291003 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:38.291063 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:38.291184 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:23:38.291449 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:38.291991 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:38.292948 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:38.293280 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:38.293389 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:38.293668 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:23:38.294195 (+ 527us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:23:38.294320 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:38.294382 (+ 62us) server_negotiation.cc:299] Negotiation successful
0811 17:23:38.294450 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 17:23:38.301622 7997 raft_consensus.cc:1261] T 37712c2f2b834bdcaf5f7d1d027e9d24 P c5a46e80c87e4bd196a970caa59611bc [term 1 FOLLOWER]: Refusing update from remote peer 411abab0828a4f97a2365026a5e0c6d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:23:38.301623 7529 raft_consensus.cc:1261] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 434f9ce8071943428076a659fd2cdfa2 [term 1 FOLLOWER]: Refusing update from remote peer 411abab0828a4f97a2365026a5e0c6d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:23:38.302444 8109 consensus_queue.cc:1035] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "434f9ce8071943428076a659fd2cdfa2" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 40345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:23:38.302745 8119 consensus_queue.cc:1035] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5a46e80c87e4bd196a970caa59611bc" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 39135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:23:38.314378 8127 mvcc.cc:205] Tried to move back new op lower bound from 7317377098954973184 to 7317377098721173504. Current Snapshot: MvccSnapshot[applied={T|T < 7317377098954973184}]
I20260811 17:23:38.320528 7282 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:23:38.320709 7282 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:23:38.323510 7282 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc033b899107487e93e3228fe88243ed: Sending DeleteTablet for 3 replicas of tablet 37712c2f2b834bdcaf5f7d1d027e9d24
I20260811 17:23:38.324541 7957 tablet_service.cc:1514] Processing DeleteTablet for tablet 37712c2f2b834bdcaf5f7d1d027e9d24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:23:38 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:44568
I20260811 17:23:38.324510 7717 tablet_service.cc:1514] Processing DeleteTablet for tablet 37712c2f2b834bdcaf5f7d1d027e9d24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:23:38 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:52334
I20260811 17:23:38.324565 7485 tablet_service.cc:1514] Processing DeleteTablet for tablet 37712c2f2b834bdcaf5f7d1d027e9d24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:23:38 UTC) from {username='kudu', principal='kudu/127.9.157.190@KRBTEST.COM'} at 127.0.0.1:52504
I20260811 17:23:38.325193 8142 tablet_replica.cc:330] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 434f9ce8071943428076a659fd2cdfa2: stopping tablet replica
I20260811 17:23:38.325604 8138 tablet_replica.cc:330] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6: stopping tablet replica
I20260811 17:23:38.325603 8142 raft_consensus.cc:2229] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 434f9ce8071943428076a659fd2cdfa2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:23:38.325942 8138 raft_consensus.cc:2229] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6 [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:23:38.326009 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 7386
I20260811 17:23:38.368710 8138 raft_consensus.cc:2258] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:23:38.371723 8138 ts_tablet_manager.cc:1905] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:23:38.373051 8141 tablet_replica.cc:330] T 37712c2f2b834bdcaf5f7d1d027e9d24 P c5a46e80c87e4bd196a970caa59611bc: stopping tablet replica
I20260811 17:23:38.373605 8141 raft_consensus.cc:2229] T 37712c2f2b834bdcaf5f7d1d027e9d24 P c5a46e80c87e4bd196a970caa59611bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:23:38.374109 8141 raft_consensus.cc:2258] T 37712c2f2b834bdcaf5f7d1d027e9d24 P c5a46e80c87e4bd196a970caa59611bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:23:38.378005 8138 ts_tablet_manager.cc:1918] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:23:38.378181 8138 log.cc:1196] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-1/wal/wals/37712c2f2b834bdcaf5f7d1d027e9d24
I20260811 17:23:38.378531 8138 ts_tablet_manager.cc:1939] T 37712c2f2b834bdcaf5f7d1d027e9d24 P 411abab0828a4f97a2365026a5e0c6d6: Deleting consensus metadata
I20260811 17:23:38.379940 7263 catalog_manager.cc:4841] TS 411abab0828a4f97a2365026a5e0c6d6 (127.9.157.130:44541): tablet 37712c2f2b834bdcaf5f7d1d027e9d24 (table test-table [id=4425fdf618e84a32a98d445b573928e7]) successfully deleted
I20260811 17:23:38.380826 8141 ts_tablet_manager.cc:1905] T 37712c2f2b834bdcaf5f7d1d027e9d24 P c5a46e80c87e4bd196a970caa59611bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:23:38.383179 8141 ts_tablet_manager.cc:1918] T 37712c2f2b834bdcaf5f7d1d027e9d24 P c5a46e80c87e4bd196a970caa59611bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:23:38.383325 8141 log.cc:1196] T 37712c2f2b834bdcaf5f7d1d027e9d24 P c5a46e80c87e4bd196a970caa59611bc: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786468743878666-9846-0/minicluster-data/ts-2/wal/wals/37712c2f2b834bdcaf5f7d1d027e9d24
I20260811 17:23:38.383726 8141 ts_tablet_manager.cc:1939] T 37712c2f2b834bdcaf5f7d1d027e9d24 P c5a46e80c87e4bd196a970caa59611bc: Deleting consensus metadata
I20260811 17:23:38.385083 7262 catalog_manager.cc:4841] TS c5a46e80c87e4bd196a970caa59611bc (127.9.157.131:39135): tablet 37712c2f2b834bdcaf5f7d1d027e9d24 (table test-table [id=4425fdf618e84a32a98d445b573928e7]) successfully deleted
W20260811 17:23:38.388345 7262 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.157.129:40345 (error 108)
I20260811 17:23:38.388484 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 7619
I20260811 17:23:38.399945 8144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:38.397945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:40345 (local address 127.0.0.1:52518)
0811 17:23:38.399597 (+ 1652us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:38.399763 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.129:40345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1514,"thread_start_us":85,"threads_started":1}
W20260811 17:23:38.400492 7262 catalog_manager.cc:4584] TS 434f9ce8071943428076a659fd2cdfa2 (127.9.157.129:40345): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 37712c2f2b834bdcaf5f7d1d027e9d24: Network error: Client connection negotiation failed: client connection to 127.9.157.129:40345: connect: Connection refused (error 111)
I20260811 17:23:38.407429 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 7849
I20260811 17:23:38.424547 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 7236
2026-08-11T17:23:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (3107 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T17:23:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:23:38Z Disabled control of system clock
I20260811 17:23:38.467347 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:34069
--webserver_interface=127.9.157.190
--webserver_port=0
--builtin_ntp_servers=127.9.157.148:42497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:34069
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 17:23:38.588529 8154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:38.588855 8154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:38.588892 8154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:38.593282 8154 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:23:38.593359 8154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:38.593374 8154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:38.593395 8154 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:23:38.593411 8154 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:23:38.598598 8154 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:42497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:34069
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:34069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:38.599946 8154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:38.601418 8154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:38.610459 8164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:38.610666 8154 server_base.cc:1034] running on GCE node
W20260811 17:23:38.610461 8166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:38.610459 8163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:38.611429 8154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:38.612675 8154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:38.613906 8154 hybrid_clock.cc:648] HybridClock initialized: now 1786469018613889 us; error 29 us; skew 500 ppm
I20260811 17:23:38.616072 8154 webserver.cc:460] Webserver started at http://127.9.157.190:36433/ using document root <none> and password file <none>
I20260811 17:23:38.616715 8154 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:38.616817 8154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:38.617069 8154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:38.619277 8154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "80ef8e421dff44bbbda35c139f8a29f5"
format_stamp: "Formatted at 2026-08-11 17:23:38 on dist-test-slave-mjjr"
server_key: "98ee2815d703359765cdabe3a83c297f"
server_key_iv: "da67db7a798c8d61be9f56cf82e1d3e0"
server_key_version: "encryptionkey@0"
I20260811 17:23:38.619926 8154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "80ef8e421dff44bbbda35c139f8a29f5"
format_stamp: "Formatted at 2026-08-11 17:23:38 on dist-test-slave-mjjr"
server_key: "98ee2815d703359765cdabe3a83c297f"
server_key_iv: "da67db7a798c8d61be9f56cf82e1d3e0"
server_key_version: "encryptionkey@0"
I20260811 17:23:38.623800 8154 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 17:23:38.626346 8176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:38.627596 8154 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 17:23:38.627777 8154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "80ef8e421dff44bbbda35c139f8a29f5"
format_stamp: "Formatted at 2026-08-11 17:23:38 on dist-test-slave-mjjr"
server_key: "98ee2815d703359765cdabe3a83c297f"
server_key_iv: "da67db7a798c8d61be9f56cf82e1d3e0"
server_key_version: "encryptionkey@0"
I20260811 17:23:38.627928 8154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:38.637799 8154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:38.638494 8154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:38.638691 8154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:38.646147 8154 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:34069
I20260811 17:23:38.646181 8256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:34069 every 8 connection(s)
I20260811 17:23:38.647199 8154 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:23:38.650523 8257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:38.654420 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 8154
I20260811 17:23:38.654529 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:23:38.654830 9846 external_mini_cluster.cc:1428] Setting key b2c4023ffd291fbd4fe781c982160355
I20260811 17:23:38.656872 8257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80ef8e421dff44bbbda35c139f8a29f5: Bootstrap starting.
I20260811 17:23:38.659803 8257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80ef8e421dff44bbbda35c139f8a29f5: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:38.660966 8257 log.cc:824] T 00000000000000000000000000000000 P 80ef8e421dff44bbbda35c139f8a29f5: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:38.663604 8257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80ef8e421dff44bbbda35c139f8a29f5: No bootstrap required, opened a new log
I20260811 17:23:38.665547 8263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:38.656376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47366 (local address 127.9.157.190:34069)
0811 17:23:38.657040 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:38.657071 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:38.657108 (+ 37us) server_negotiation.cc:407] Connection header received
0811 17:23:38.657742 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:38.657758 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:38.658153 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:38.658441 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:38.658801 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:38.660422 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:38.661789 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:38.662593 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:38.662718 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:38.663681 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:38.663694 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:38.663715 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:38.664173 (+ 458us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:38.664230 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:38.664252 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:38.664329 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:38.664523 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 17:23:38.664824 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":138,"threads_started":1}
I20260811 17:23:38.667693 8257 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80ef8e421dff44bbbda35c139f8a29f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80ef8e421dff44bbbda35c139f8a29f5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34069 } }
I20260811 17:23:38.668023 8257 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80ef8e421dff44bbbda35c139f8a29f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:38.668114 8257 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80ef8e421dff44bbbda35c139f8a29f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80ef8e421dff44bbbda35c139f8a29f5, State: Initialized, Role: FOLLOWER
I20260811 17:23:38.668557 8257 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80ef8e421dff44bbbda35c139f8a29f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80ef8e421dff44bbbda35c139f8a29f5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34069 } }
I20260811 17:23:38.668720 8257 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80ef8e421dff44bbbda35c139f8a29f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:23:38.668794 8257 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80ef8e421dff44bbbda35c139f8a29f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:23:38.668926 8257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80ef8e421dff44bbbda35c139f8a29f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:38.669999 8257 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80ef8e421dff44bbbda35c139f8a29f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80ef8e421dff44bbbda35c139f8a29f5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34069 } }
I20260811 17:23:38.670377 8257 leader_election.cc:302] T 00000000000000000000000000000000 P 80ef8e421dff44bbbda35c139f8a29f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80ef8e421dff44bbbda35c139f8a29f5; no voters:
I20260811 17:23:38.670763 8257 leader_election.cc:288] T 00000000000000000000000000000000 P 80ef8e421dff44bbbda35c139f8a29f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:23:38.670890 8266 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80ef8e421dff44bbbda35c139f8a29f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:23:38.671168 8266 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80ef8e421dff44bbbda35c139f8a29f5 [term 1 LEADER]: Becoming Leader. State: Replica: 80ef8e421dff44bbbda35c139f8a29f5, State: Running, Role: LEADER
I20260811 17:23:38.671486 8266 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80ef8e421dff44bbbda35c139f8a29f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80ef8e421dff44bbbda35c139f8a29f5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34069 } }
I20260811 17:23:38.672246 8257 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80ef8e421dff44bbbda35c139f8a29f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:23:38.673600 8271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80ef8e421dff44bbbda35c139f8a29f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80ef8e421dff44bbbda35c139f8a29f5. Latest consensus state: current_term: 1 leader_uuid: "80ef8e421dff44bbbda35c139f8a29f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80ef8e421dff44bbbda35c139f8a29f5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34069 } } }
I20260811 17:23:38.673748 8271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80ef8e421dff44bbbda35c139f8a29f5 [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:38.674221 8267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80ef8e421dff44bbbda35c139f8a29f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80ef8e421dff44bbbda35c139f8a29f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80ef8e421dff44bbbda35c139f8a29f5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 34069 } } }
I20260811 17:23:38.674965 8284 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:23:38.675182 8267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80ef8e421dff44bbbda35c139f8a29f5 [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:38.678112 8284 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:23:38.684580 8284 catalog_manager.cc:1347] Generated new cluster ID: 12a9ff70f2244e39b3343e15645a6156
I20260811 17:23:38.684680 8284 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:23:38.712555 8284 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:23:38.713961 8284 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:23:38.722662 8284 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80ef8e421dff44bbbda35c139f8a29f5: Generated new TSK 0
I20260811 17:23:38.723639 8284 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:23:38.739957 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.129:0
--local_ip_for_outbound_sockets=127.9.157.129
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:34069
--builtin_ntp_servers=127.9.157.148:42497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 17:23:38.864590 8308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:38.865058 8308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:38.865177 8308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:38.869516 8308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:38.869581 8308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:38.869639 8308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.129
I20260811 17:23:38.875119 8308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:42497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.157.129
--webserver_port=0
--tserver_master_addrs=127.9.157.190:34069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.129
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:38.876433 8308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:38.877916 8308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:38.888608 8316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:38.888608 8315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:38.888612 8322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:38.889101 8308 server_base.cc:1034] running on GCE node
I20260811 17:23:38.889566 8308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:38.890407 8308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:38.891633 8308 hybrid_clock.cc:648] HybridClock initialized: now 1786469018891598 us; error 45 us; skew 500 ppm
I20260811 17:23:38.893857 8308 webserver.cc:460] Webserver started at http://127.9.157.129:46153/ using document root <none> and password file <none>
I20260811 17:23:38.894500 8308 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:38.894574 8308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:38.894771 8308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:38.897152 8308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-0/data/instance:
uuid: "7c20503843b04322b2a50cc62ee2666f"
format_stamp: "Formatted at 2026-08-11 17:23:38 on dist-test-slave-mjjr"
server_key: "c800ece25f51870459f96efa2b31fbe0"
server_key_iv: "a23c49400baf249c728aa69c1c43e86b"
server_key_version: "encryptionkey@0"
I20260811 17:23:38.897857 8308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance:
uuid: "7c20503843b04322b2a50cc62ee2666f"
format_stamp: "Formatted at 2026-08-11 17:23:38 on dist-test-slave-mjjr"
server_key: "c800ece25f51870459f96efa2b31fbe0"
server_key_iv: "a23c49400baf249c728aa69c1c43e86b"
server_key_version: "encryptionkey@0"
I20260811 17:23:38.902217 8308 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:23:38.905153 8334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:38.906595 8308 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:23:38.906721 8308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-0/wal
uuid: "7c20503843b04322b2a50cc62ee2666f"
format_stamp: "Formatted at 2026-08-11 17:23:38 on dist-test-slave-mjjr"
server_key: "c800ece25f51870459f96efa2b31fbe0"
server_key_iv: "a23c49400baf249c728aa69c1c43e86b"
server_key_version: "encryptionkey@0"
I20260811 17:23:38.906915 8308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:38.922859 8308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:38.923780 8308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:38.924012 8308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:38.924710 8308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:23:38.925962 8308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:23:38.926018 8308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:38.926090 8308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:23:38.926123 8308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:38.937412 8308 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.129:36835
I20260811 17:23:38.937448 8541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.129:36835 every 8 connection(s)
I20260811 17:23:38.938668 8308 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-0/data/info.pb
I20260811 17:23:38.948937 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 8308
I20260811 17:23:38.949137 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-0/wal/instance
I20260811 17:23:38.949316 8263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:38.940917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.129:39351 (local address 127.9.157.190:34069)
0811 17:23:38.941099 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:38.941105 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:38.942422 (+ 1317us) server_negotiation.cc:407] Connection header received
0811 17:23:38.944012 (+ 1590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:38.944015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:38.944068 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:38.944154 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:38.945184 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:38.945774 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:38.946985 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:38.947217 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:38.947321 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:38.948696 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:38.948703 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:38.948705 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:38.948724 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:38.948750 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:38.948764 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:38.948856 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:38.949041 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 17:23:38.949130 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 17:23:38.949504 9846 external_mini_cluster.cc:1428] Setting key e22ac6c8757bad2e73d344d0011bd1ca
I20260811 17:23:38.950397 8547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:38.941465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34069 (local address 127.9.157.129:39351)
0811 17:23:38.942236 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:38.942285 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:38.943749 (+ 1464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:38.944343 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:38.944351 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:38.944443 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:23:38.945014 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:38.945028 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:38.945899 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:38.945902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:38.946835 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:38.946845 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:38.947389 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:38.947393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:38.947560 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:38.948334 (+ 774us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:23:38.948336 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:23:38.948383 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:23:38.948462 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:23:38.948872 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:23:38.948881 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:23:38.948886 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 17:23:38.949002 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 17:23:38.949383 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":142,"threads_started":1}
I20260811 17:23:38.951936 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.130:0
--local_ip_for_outbound_sockets=127.9.157.130
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:34069
--builtin_ntp_servers=127.9.157.148:42497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 17:23:38.951977 8543 heartbeater.cc:347] Connected to a master server at 127.9.157.190:34069
I20260811 17:23:38.952296 8543 heartbeater.cc:464] Registering TS with master...
I20260811 17:23:38.952898 8543 heartbeater.cc:511] Master 127.9.157.190:34069 requested a full tablet report, sending...
I20260811 17:23:38.954770 8199 ts_manager.cc:194] Registered new tserver with Master: 7c20503843b04322b2a50cc62ee2666f (127.9.157.129:36835)
I20260811 17:23:38.956260 8199 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.157.129:39351
W20260811 17:23:39.073489 8548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:39.073887 8548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:39.074119 8548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:39.078572 8548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:39.078702 8548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:39.078810 8548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.130
I20260811 17:23:39.083990 8548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:42497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.157.130
--webserver_port=0
--tserver_master_addrs=127.9.157.190:34069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.130
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:39.085286 8548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:39.086879 8548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:39.097208 8560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:39.097358 8548 server_base.cc:1034] running on GCE node
W20260811 17:23:39.097278 8562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:39.097555 8558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:39.098037 8548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:39.098781 8548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:39.099982 8548 hybrid_clock.cc:648] HybridClock initialized: now 1786469019099944 us; error 49 us; skew 500 ppm
I20260811 17:23:39.101976 8548 webserver.cc:460] Webserver started at http://127.9.157.130:39031/ using document root <none> and password file <none>
I20260811 17:23:39.102605 8548 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:39.102672 8548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:39.102914 8548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:39.105209 8548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-1/data/instance:
uuid: "14d440c20aa24e0796bb55a07b4d3840"
format_stamp: "Formatted at 2026-08-11 17:23:39 on dist-test-slave-mjjr"
server_key: "d6e82e8393b6652ec904da2563728002"
server_key_iv: "3fdfdd5992ca4a942e801035ebd8b60e"
server_key_version: "encryptionkey@0"
I20260811 17:23:39.105880 8548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance:
uuid: "14d440c20aa24e0796bb55a07b4d3840"
format_stamp: "Formatted at 2026-08-11 17:23:39 on dist-test-slave-mjjr"
server_key: "d6e82e8393b6652ec904da2563728002"
server_key_iv: "3fdfdd5992ca4a942e801035ebd8b60e"
server_key_version: "encryptionkey@0"
I20260811 17:23:39.110030 8548 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:23:39.113157 8575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:39.114710 8548 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 17:23:39.114876 8548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-1/wal
uuid: "14d440c20aa24e0796bb55a07b4d3840"
format_stamp: "Formatted at 2026-08-11 17:23:39 on dist-test-slave-mjjr"
server_key: "d6e82e8393b6652ec904da2563728002"
server_key_iv: "3fdfdd5992ca4a942e801035ebd8b60e"
server_key_version: "encryptionkey@0"
I20260811 17:23:39.115013 8548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:39.132115 8548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:39.132925 8548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:39.133147 8548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:39.133822 8548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:23:39.134876 8548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:23:39.134929 8548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:39.134971 8548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:23:39.135027 8548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:39.145291 8548 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.130:35331
I20260811 17:23:39.145386 8775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.130:35331 every 8 connection(s)
I20260811 17:23:39.146350 8548 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-1/data/info.pb
I20260811 17:23:39.150646 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 8548
I20260811 17:23:39.150822 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-1/wal/instance
I20260811 17:23:39.151250 9846 external_mini_cluster.cc:1428] Setting key fcc204a9b99c4f04e32ef00f4958aa28
I20260811 17:23:39.153446 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.157.131:0
--local_ip_for_outbound_sockets=127.9.157.131
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:34069
--builtin_ntp_servers=127.9.157.148:42497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 17:23:39.157200 8263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:39.148495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:39989 (local address 127.9.157.190:34069)
0811 17:23:39.148646 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:39.148650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:39.149760 (+ 1110us) server_negotiation.cc:407] Connection header received
0811 17:23:39.151611 (+ 1851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:39.151614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:39.151718 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:39.151826 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:39.153002 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.153566 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.154820 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.155054 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.155170 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:39.156600 (+ 1430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:39.156604 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:39.156606 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:39.156621 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:39.156643 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:39.156657 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:39.156767 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:39.156943 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 17:23:39.157040 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 17:23:39.158219 8778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:39.148902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34069 (local address 127.9.157.130:39989)
0811 17:23:39.149596 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:39.149639 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:39.151347 (+ 1708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:39.152044 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:39.152053 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:39.152173 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:23:39.152843 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:39.152866 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.153742 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.153744 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:39.154671 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:39.154682 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.155227 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.155232 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:39.155422 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:39.156262 (+ 840us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:23:39.156264 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:23:39.156313 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:23:39.156391 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:23:39.156757 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:23:39.156769 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:23:39.156777 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 17:23:39.156930 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 17:23:39.157281 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":124,"threads_started":1}
I20260811 17:23:39.159610 8776 heartbeater.cc:347] Connected to a master server at 127.9.157.190:34069
I20260811 17:23:39.159976 8776 heartbeater.cc:464] Registering TS with master...
I20260811 17:23:39.160794 8776 heartbeater.cc:511] Master 127.9.157.190:34069 requested a full tablet report, sending...
I20260811 17:23:39.162232 8199 ts_manager.cc:194] Registered new tserver with Master: 14d440c20aa24e0796bb55a07b4d3840 (127.9.157.130:35331)
I20260811 17:23:39.162873 8199 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.157.130:39989
W20260811 17:23:39.283475 8779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:39.283828 8779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:39.283903 8779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:39.288196 8779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:39.288267 8779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:39.288367 8779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.157.131
I20260811 17:23:39.293524 8779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:42497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.157.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.157.131
--webserver_port=0
--tserver_master_addrs=127.9.157.190:34069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.157.131
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:39.294755 8779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:39.296274 8779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:39.306715 8785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:39.306751 8791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:39.306710 8784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:39.306933 8779 server_base.cc:1034] running on GCE node
I20260811 17:23:39.307729 8779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:39.308526 8779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:39.309741 8779 hybrid_clock.cc:648] HybridClock initialized: now 1786469019309725 us; error 32 us; skew 500 ppm
I20260811 17:23:39.312083 8779 webserver.cc:460] Webserver started at http://127.9.157.131:41527/ using document root <none> and password file <none>
I20260811 17:23:39.312762 8779 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:39.312862 8779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:39.313130 8779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:39.315864 8779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-2/data/instance:
uuid: "94bb023a08d147348cbbca4c7ae187e6"
format_stamp: "Formatted at 2026-08-11 17:23:39 on dist-test-slave-mjjr"
server_key: "ee9577b25c64644b6851e4aaed180945"
server_key_iv: "e38e846bd5dc6929bb6816cb5c53a142"
server_key_version: "encryptionkey@0"
I20260811 17:23:39.316535 8779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance:
uuid: "94bb023a08d147348cbbca4c7ae187e6"
format_stamp: "Formatted at 2026-08-11 17:23:39 on dist-test-slave-mjjr"
server_key: "ee9577b25c64644b6851e4aaed180945"
server_key_iv: "e38e846bd5dc6929bb6816cb5c53a142"
server_key_version: "encryptionkey@0"
I20260811 17:23:39.322402 8779 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 17:23:39.325625 8800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:39.327196 8779 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:23:39.327386 8779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-2/wal
uuid: "94bb023a08d147348cbbca4c7ae187e6"
format_stamp: "Formatted at 2026-08-11 17:23:39 on dist-test-slave-mjjr"
server_key: "ee9577b25c64644b6851e4aaed180945"
server_key_iv: "e38e846bd5dc6929bb6816cb5c53a142"
server_key_version: "encryptionkey@0"
I20260811 17:23:39.327538 8779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:39.342840 8779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:39.343575 8779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:39.343829 8779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:39.344471 8779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 17:23:39.345489 8779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 17:23:39.345538 8779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:39.345582 8779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 17:23:39.345600 8779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:39.355553 8779 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.131:38359
I20260811 17:23:39.355723 8984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.131:38359 every 8 connection(s)
I20260811 17:23:39.356707 8779 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-2/data/info.pb
I20260811 17:23:39.362967 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 8779
I20260811 17:23:39.363087 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-2/wal/instance
I20260811 17:23:39.363428 9846 external_mini_cluster.cc:1428] Setting key c4bf5d98764e4e61427bce80c732236f
I20260811 17:23:39.368036 8263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:39.359014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.131:34461 (local address 127.9.157.190:34069)
0811 17:23:39.359221 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:39.359226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:39.360512 (+ 1286us) server_negotiation.cc:407] Connection header received
0811 17:23:39.362097 (+ 1585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:39.362101 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:39.362170 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:39.362284 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:39.363401 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.364095 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.365852 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.366216 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.366327 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:39.367473 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:39.367477 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:39.367479 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:39.367493 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:39.367515 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:39.367523 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:39.367583 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:39.367835 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 17:23:39.367914 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 17:23:39.368952 8987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:39.359453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:34069 (local address 127.9.157.131:34461)
0811 17:23:39.360322 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:39.360372 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:39.361836 (+ 1464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:39.362506 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:39.362514 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:39.362608 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:23:39.363241 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:39.363259 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.364262 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.364266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:39.365712 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:39.365726 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.366388 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.366391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:39.366564 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:39.367199 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 17:23:39.367201 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:23:39.367247 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 17:23:39.367324 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:23:39.367601 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:23:39.367609 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:23:39.367614 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 17:23:39.367784 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 17:23:39.368063 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":224,"threads_started":1}
I20260811 17:23:39.370468 8985 heartbeater.cc:347] Connected to a master server at 127.9.157.190:34069
I20260811 17:23:39.370792 8985 heartbeater.cc:464] Registering TS with master...
I20260811 17:23:39.371395 8985 heartbeater.cc:511] Master 127.9.157.190:34069 requested a full tablet report, sending...
I20260811 17:23:39.372805 8199 ts_manager.cc:194] Registered new tserver with Master: 94bb023a08d147348cbbca4c7ae187e6 (127.9.157.131:38359)
I20260811 17:23:39.373481 8199 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.157.131:34461
I20260811 17:23:39.378216 9846 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 17:23:39.384298 8263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:39.380258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47382 (local address 127.9.157.190:34069)
0811 17:23:39.380463 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:39.380467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:39.380569 (+ 102us) server_negotiation.cc:407] Connection header received
0811 17:23:39.380673 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:39.380675 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:39.380719 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:39.380790 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:39.381028 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.381571 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.382433 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.382653 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.382739 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:39.383617 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:39.383621 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:39.383623 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:39.383638 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:39.383734 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:39.383779 (+ 45us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:39.383860 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:39.383977 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 17:23:39.384075 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 17:23:39.387858 8199 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47382:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 17:23:39.391077 8199 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 17:23:39.407707 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:39.401944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42112 (local address 127.9.157.129:36835)
0811 17:23:39.402566 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:39.402580 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:39.402608 (+ 28us) server_negotiation.cc:407] Connection header received
0811 17:23:39.402715 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:39.402721 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:39.402914 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:39.403080 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:39.403562 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.404921 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.406301 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.406883 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.407016 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:39.407266 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:39.407368 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 17:23:39.407496 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":165,"threads_started":1}
I20260811 17:23:39.407701 9001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:39.400943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:38359 (local address 127.0.0.1:42430)
0811 17:23:39.401625 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:39.401686 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:39.401827 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:39.402931 (+ 1104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:39.402938 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:39.402964 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:39.403145 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:39.403154 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.404694 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.404698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:39.406051 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:39.406063 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.407139 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.407144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:39.407256 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:39.407286 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 17:23:39.407351 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 17:23:39.407536 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":83,"thread_start_us":147,"threads_started":1}
I20260811 17:23:39.408052 8997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:39.401125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:36835 (local address 127.0.0.1:42112)
0811 17:23:39.402318 (+ 1193us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:39.402333 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:39.402434 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:39.403253 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:39.403258 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:39.403281 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:39.403410 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:39.403416 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.405050 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.405053 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:39.406168 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:39.406178 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.407032 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.407036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:39.407150 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:39.407200 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 17:23:39.407326 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 17:23:39.407938 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1131,"mutex_wait_us":59,"thread_start_us":107,"threads_started":1}
I20260811 17:23:39.408691 8999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:39.401056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45844 (local address 127.9.157.130:35331)
0811 17:23:39.401552 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:39.401565 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:39.402555 (+ 990us) server_negotiation.cc:407] Connection header received
0811 17:23:39.402722 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:39.402727 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:39.402905 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:39.403040 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:39.403411 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.404339 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.406021 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.406950 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.407100 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:39.407341 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:39.407755 (+ 414us) server_negotiation.cc:299] Negotiation successful
0811 17:23:39.408495 (+ 740us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":124,"threads_started":1}
I20260811 17:23:39.408705 9005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:39.401865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42430 (local address 127.9.157.131:38359)
0811 17:23:39.402335 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:39.402348 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:39.402377 (+ 29us) server_negotiation.cc:407] Connection header received
0811 17:23:39.402463 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:39.402471 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:39.402735 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:39.402914 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:39.403283 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.404556 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.406200 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.406985 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.407539 (+ 554us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:39.408334 (+ 795us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:39.408402 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 17:23:39.408499 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":159,"threads_started":1}
I20260811 17:23:39.409360 9007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:39.400943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.130:35331 (local address 127.0.0.1:45844)
0811 17:23:39.402435 (+ 1492us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:39.402453 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:39.402570 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:39.403105 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:39.403108 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:39.403125 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:39.403264 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:39.403271 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.404532 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.404536 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:39.405862 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:39.405873 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.407416 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.407423 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:39.407528 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:39.407552 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 17:23:39.409099 (+ 1547us) client_negotiation.cc:240] Negotiation successful
0811 17:23:39.409207 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1406,"mutex_wait_us":1023,"thread_start_us":124,"threads_started":1}
I20260811 17:23:39.411558 8879 tablet_service.cc:1467] Processing CreateTablet for tablet 794a9c93dbab4767a1c21fca8bc1f458 (DEFAULT_TABLE table=test-table [id=7434468cee6e4ce89fcb2be8521b39c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:23:39.411559 8423 tablet_service.cc:1467] Processing CreateTablet for tablet 794a9c93dbab4767a1c21fca8bc1f458 (DEFAULT_TABLE table=test-table [id=7434468cee6e4ce89fcb2be8521b39c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:23:39.412140 8665 tablet_service.cc:1467] Processing CreateTablet for tablet 794a9c93dbab4767a1c21fca8bc1f458 (DEFAULT_TABLE table=test-table [id=7434468cee6e4ce89fcb2be8521b39c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 17:23:39.412899 8423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 794a9c93dbab4767a1c21fca8bc1f458. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:39.412986 8879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 794a9c93dbab4767a1c21fca8bc1f458. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:39.413444 8665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 794a9c93dbab4767a1c21fca8bc1f458. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:39.421038 9008 tablet_bootstrap.cc:492] T 794a9c93dbab4767a1c21fca8bc1f458 P 7c20503843b04322b2a50cc62ee2666f: Bootstrap starting.
I20260811 17:23:39.421995 9013 tablet_bootstrap.cc:492] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840: Bootstrap starting.
I20260811 17:23:39.422420 9010 tablet_bootstrap.cc:492] T 794a9c93dbab4767a1c21fca8bc1f458 P 94bb023a08d147348cbbca4c7ae187e6: Bootstrap starting.
I20260811 17:23:39.424443 9013 tablet_bootstrap.cc:654] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:39.424449 9008 tablet_bootstrap.cc:654] T 794a9c93dbab4767a1c21fca8bc1f458 P 7c20503843b04322b2a50cc62ee2666f: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:39.424443 9010 tablet_bootstrap.cc:654] T 794a9c93dbab4767a1c21fca8bc1f458 P 94bb023a08d147348cbbca4c7ae187e6: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:39.425289 9010 log.cc:824] T 794a9c93dbab4767a1c21fca8bc1f458 P 94bb023a08d147348cbbca4c7ae187e6: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:39.425416 9013 log.cc:824] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:39.425416 9008 log.cc:824] T 794a9c93dbab4767a1c21fca8bc1f458 P 7c20503843b04322b2a50cc62ee2666f: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:39.427531 9010 tablet_bootstrap.cc:492] T 794a9c93dbab4767a1c21fca8bc1f458 P 94bb023a08d147348cbbca4c7ae187e6: No bootstrap required, opened a new log
I20260811 17:23:39.427812 9010 ts_tablet_manager.cc:1397] T 794a9c93dbab4767a1c21fca8bc1f458 P 94bb023a08d147348cbbca4c7ae187e6: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 17:23:39.427810 9008 tablet_bootstrap.cc:492] T 794a9c93dbab4767a1c21fca8bc1f458 P 7c20503843b04322b2a50cc62ee2666f: No bootstrap required, opened a new log
I20260811 17:23:39.427860 9013 tablet_bootstrap.cc:492] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840: No bootstrap required, opened a new log
I20260811 17:23:39.428020 9013 ts_tablet_manager.cc:1397] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 17:23:39.428020 9008 ts_tablet_manager.cc:1397] T 794a9c93dbab4767a1c21fca8bc1f458 P 7c20503843b04322b2a50cc62ee2666f: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 17:23:39.431905 9010 raft_consensus.cc:348] T 794a9c93dbab4767a1c21fca8bc1f458 P 94bb023a08d147348cbbca4c7ae187e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d440c20aa24e0796bb55a07b4d3840" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35331 } } peers { permanent_uuid: "94bb023a08d147348cbbca4c7ae187e6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 38359 } } peers { permanent_uuid: "7c20503843b04322b2a50cc62ee2666f" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 36835 } }
I20260811 17:23:39.432487 9010 raft_consensus.cc:374] T 794a9c93dbab4767a1c21fca8bc1f458 P 94bb023a08d147348cbbca4c7ae187e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:39.432557 9010 raft_consensus.cc:729] T 794a9c93dbab4767a1c21fca8bc1f458 P 94bb023a08d147348cbbca4c7ae187e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94bb023a08d147348cbbca4c7ae187e6, State: Initialized, Role: FOLLOWER
I20260811 17:23:39.432641 9008 raft_consensus.cc:348] T 794a9c93dbab4767a1c21fca8bc1f458 P 7c20503843b04322b2a50cc62ee2666f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d440c20aa24e0796bb55a07b4d3840" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35331 } } peers { permanent_uuid: "94bb023a08d147348cbbca4c7ae187e6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 38359 } } peers { permanent_uuid: "7c20503843b04322b2a50cc62ee2666f" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 36835 } }
I20260811 17:23:39.432641 9013 raft_consensus.cc:348] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d440c20aa24e0796bb55a07b4d3840" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35331 } } peers { permanent_uuid: "94bb023a08d147348cbbca4c7ae187e6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 38359 } } peers { permanent_uuid: "7c20503843b04322b2a50cc62ee2666f" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 36835 } }
I20260811 17:23:39.432940 9013 raft_consensus.cc:374] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:39.432940 9008 raft_consensus.cc:374] T 794a9c93dbab4767a1c21fca8bc1f458 P 7c20503843b04322b2a50cc62ee2666f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:39.433007 9013 raft_consensus.cc:729] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14d440c20aa24e0796bb55a07b4d3840, State: Initialized, Role: FOLLOWER
I20260811 17:23:39.433007 9008 raft_consensus.cc:729] T 794a9c93dbab4767a1c21fca8bc1f458 P 7c20503843b04322b2a50cc62ee2666f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c20503843b04322b2a50cc62ee2666f, State: Initialized, Role: FOLLOWER
I20260811 17:23:39.433179 9010 consensus_queue.cc:260] T 794a9c93dbab4767a1c21fca8bc1f458 P 94bb023a08d147348cbbca4c7ae187e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d440c20aa24e0796bb55a07b4d3840" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35331 } } peers { permanent_uuid: "94bb023a08d147348cbbca4c7ae187e6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 38359 } } peers { permanent_uuid: "7c20503843b04322b2a50cc62ee2666f" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 36835 } }
I20260811 17:23:39.433468 9013 consensus_queue.cc:260] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d440c20aa24e0796bb55a07b4d3840" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35331 } } peers { permanent_uuid: "94bb023a08d147348cbbca4c7ae187e6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 38359 } } peers { permanent_uuid: "7c20503843b04322b2a50cc62ee2666f" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 36835 } }
I20260811 17:23:39.433472 9008 consensus_queue.cc:260] T 794a9c93dbab4767a1c21fca8bc1f458 P 7c20503843b04322b2a50cc62ee2666f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d440c20aa24e0796bb55a07b4d3840" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35331 } } peers { permanent_uuid: "94bb023a08d147348cbbca4c7ae187e6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 38359 } } peers { permanent_uuid: "7c20503843b04322b2a50cc62ee2666f" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 36835 } }
I20260811 17:23:39.434124 8985 heartbeater.cc:503] Master 127.9.157.190:34069 was elected leader, sending a full tablet report...
I20260811 17:23:39.434439 9010 ts_tablet_manager.cc:1428] T 794a9c93dbab4767a1c21fca8bc1f458 P 94bb023a08d147348cbbca4c7ae187e6: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 17:23:39.434530 9013 ts_tablet_manager.cc:1428] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 17:23:39.434899 8776 heartbeater.cc:503] Master 127.9.157.190:34069 was elected leader, sending a full tablet report...
I20260811 17:23:39.435357 8543 heartbeater.cc:503] Master 127.9.157.190:34069 was elected leader, sending a full tablet report...
I20260811 17:23:39.435851 9008 ts_tablet_manager.cc:1428] T 794a9c93dbab4767a1c21fca8bc1f458 P 7c20503843b04322b2a50cc62ee2666f: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
W20260811 17:23:39.440009 8546 tablet.cc:2369] T 794a9c93dbab4767a1c21fca8bc1f458 P 7c20503843b04322b2a50cc62ee2666f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:23:39.607947 8986 tablet.cc:2369] T 794a9c93dbab4767a1c21fca8bc1f458 P 94bb023a08d147348cbbca4c7ae187e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:23:39.645361 9022 raft_consensus.cc:482] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 17:23:39.645637 9022 raft_consensus.cc:504] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d440c20aa24e0796bb55a07b4d3840" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35331 } } peers { permanent_uuid: "94bb023a08d147348cbbca4c7ae187e6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 38359 } } peers { permanent_uuid: "7c20503843b04322b2a50cc62ee2666f" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 36835 } }
I20260811 17:23:39.646984 9022 leader_election.cc:288] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94bb023a08d147348cbbca4c7ae187e6 (127.9.157.131:38359), 7c20503843b04322b2a50cc62ee2666f (127.9.157.129:36835)
W20260811 17:23:39.647606 8777 tablet.cc:2369] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:23:39.652187 8778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:39.647229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.131:38359 (local address 127.9.157.130:56817)
0811 17:23:39.647381 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:39.647399 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:39.647522 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:39.648574 (+ 1052us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:39.648577 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:39.648592 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:39.648702 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:39.648707 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.649439 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.649446 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:39.650969 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:39.650981 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.651717 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.651720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:39.651834 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:39.651867 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 17:23:39.651940 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 17:23:39.652014 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":133}
I20260811 17:23:39.652220 9005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:39.648191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:56817 (local address 127.9.157.131:38359)
0811 17:23:39.648355 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:39.648360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:39.648378 (+ 18us) server_negotiation.cc:407] Connection header received
0811 17:23:39.648432 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:39.648435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:39.648491 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:39.648559 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:39.648824 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.649307 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.651132 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.651550 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.651717 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:39.651775 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:39.652007 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 17:23:39.652092 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 17:23:39.652366 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:39.647747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.157.130:48747 (local address 127.9.157.129:36835)
0811 17:23:39.647890 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:39.647895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:39.647913 (+ 18us) server_negotiation.cc:407] Connection header received
0811 17:23:39.647971 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:39.647974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:39.648025 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:39.648102 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 17:23:39.648563 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.649361 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.651124 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.651751 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.651885 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:39.651933 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:39.652141 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 17:23:39.652240 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 17:23:39.652752 9026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:39.647249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:36835 (local address 127.9.157.130:48747)
0811 17:23:39.647775 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:39.647790 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:23:39.647875 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:23:39.648260 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:23:39.648264 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:23:39.648285 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 17:23:39.648417 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:39.648424 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.649477 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.649480 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:39.650969 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:23:39.650981 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.651897 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.651903 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:23:39.651998 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:39.652016 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 17:23:39.652527 (+ 511us) client_negotiation.cc:240] Negotiation successful
0811 17:23:39.652603 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"mutex_wait_us":263,"thread_start_us":108,"threads_started":1}
I20260811 17:23:39.653044 8915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "794a9c93dbab4767a1c21fca8bc1f458" candidate_uuid: "14d440c20aa24e0796bb55a07b4d3840" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94bb023a08d147348cbbca4c7ae187e6" is_pre_election: true
I20260811 17:23:39.653422 8915 raft_consensus.cc:2454] T 794a9c93dbab4767a1c21fca8bc1f458 P 94bb023a08d147348cbbca4c7ae187e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14d440c20aa24e0796bb55a07b4d3840 in term 0.
I20260811 17:23:39.653426 8460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "794a9c93dbab4767a1c21fca8bc1f458" candidate_uuid: "14d440c20aa24e0796bb55a07b4d3840" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c20503843b04322b2a50cc62ee2666f" is_pre_election: true
I20260811 17:23:39.653728 8460 raft_consensus.cc:2454] T 794a9c93dbab4767a1c21fca8bc1f458 P 7c20503843b04322b2a50cc62ee2666f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14d440c20aa24e0796bb55a07b4d3840 in term 0.
I20260811 17:23:39.654044 8578 leader_election.cc:302] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14d440c20aa24e0796bb55a07b4d3840, 94bb023a08d147348cbbca4c7ae187e6; no voters:
I20260811 17:23:39.654426 9022 raft_consensus.cc:2781] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 17:23:39.654568 9022 raft_consensus.cc:482] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 17:23:39.654660 9022 raft_consensus.cc:3037] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:39.655864 9022 raft_consensus.cc:504] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d440c20aa24e0796bb55a07b4d3840" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35331 } } peers { permanent_uuid: "94bb023a08d147348cbbca4c7ae187e6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 38359 } } peers { permanent_uuid: "7c20503843b04322b2a50cc62ee2666f" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 36835 } }
I20260811 17:23:39.656591 9022 leader_election.cc:288] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840 [CANDIDATE]: Term 1 election: Requested vote from peers 94bb023a08d147348cbbca4c7ae187e6 (127.9.157.131:38359), 7c20503843b04322b2a50cc62ee2666f (127.9.157.129:36835)
I20260811 17:23:39.656663 8460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "794a9c93dbab4767a1c21fca8bc1f458" candidate_uuid: "14d440c20aa24e0796bb55a07b4d3840" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c20503843b04322b2a50cc62ee2666f"
I20260811 17:23:39.656672 8915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "794a9c93dbab4767a1c21fca8bc1f458" candidate_uuid: "14d440c20aa24e0796bb55a07b4d3840" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94bb023a08d147348cbbca4c7ae187e6"
I20260811 17:23:39.656792 8460 raft_consensus.cc:3037] T 794a9c93dbab4767a1c21fca8bc1f458 P 7c20503843b04322b2a50cc62ee2666f [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:39.656843 8915 raft_consensus.cc:3037] T 794a9c93dbab4767a1c21fca8bc1f458 P 94bb023a08d147348cbbca4c7ae187e6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:39.657872 8915 raft_consensus.cc:2454] T 794a9c93dbab4767a1c21fca8bc1f458 P 94bb023a08d147348cbbca4c7ae187e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14d440c20aa24e0796bb55a07b4d3840 in term 1.
I20260811 17:23:39.658083 8460 raft_consensus.cc:2454] T 794a9c93dbab4767a1c21fca8bc1f458 P 7c20503843b04322b2a50cc62ee2666f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14d440c20aa24e0796bb55a07b4d3840 in term 1.
I20260811 17:23:39.658243 8578 leader_election.cc:302] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14d440c20aa24e0796bb55a07b4d3840, 94bb023a08d147348cbbca4c7ae187e6; no voters:
I20260811 17:23:39.658551 9022 raft_consensus.cc:2781] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:23:39.658806 9022 raft_consensus.cc:686] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840 [term 1 LEADER]: Becoming Leader. State: Replica: 14d440c20aa24e0796bb55a07b4d3840, State: Running, Role: LEADER
I20260811 17:23:39.659196 9022 consensus_queue.cc:237] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d440c20aa24e0796bb55a07b4d3840" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35331 } } peers { permanent_uuid: "94bb023a08d147348cbbca4c7ae187e6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 38359 } } peers { permanent_uuid: "7c20503843b04322b2a50cc62ee2666f" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 36835 } }
I20260811 17:23:39.663004 8199 catalog_manager.cc:5495] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840 reported cstate change: term changed from 0 to 1, leader changed from <none> to 14d440c20aa24e0796bb55a07b4d3840 (127.9.157.130). New cstate: current_term: 1 leader_uuid: "14d440c20aa24e0796bb55a07b4d3840" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d440c20aa24e0796bb55a07b4d3840" member_type: VOTER last_known_addr { host: "127.9.157.130" port: 35331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94bb023a08d147348cbbca4c7ae187e6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 38359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c20503843b04322b2a50cc62ee2666f" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 36835 } health_report { overall_health: UNKNOWN } } }
I20260811 17:23:39.704600 8999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:39.700990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45854 (local address 127.9.157.130:35331)
0811 17:23:39.701123 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:39.701127 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:39.701141 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:23:39.701222 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:39.701224 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:39.701283 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:39.701384 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:23:39.701640 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.702158 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.703016 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:39.703314 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:39.703401 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:39.703746 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:23:39.704213 (+ 467us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:23:39.704312 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:39.704387 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 17:23:39.704441 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 17:23:39.711076 8460 raft_consensus.cc:1261] T 794a9c93dbab4767a1c21fca8bc1f458 P 7c20503843b04322b2a50cc62ee2666f [term 1 FOLLOWER]: Refusing update from remote peer 14d440c20aa24e0796bb55a07b4d3840: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:23:39.711076 8915 raft_consensus.cc:1261] T 794a9c93dbab4767a1c21fca8bc1f458 P 94bb023a08d147348cbbca4c7ae187e6 [term 1 FOLLOWER]: Refusing update from remote peer 14d440c20aa24e0796bb55a07b4d3840: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:23:39.711979 9022 consensus_queue.cc:1035] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c20503843b04322b2a50cc62ee2666f" member_type: VOTER last_known_addr { host: "127.9.157.129" port: 36835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:23:39.712255 9028 consensus_queue.cc:1035] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94bb023a08d147348cbbca4c7ae187e6" member_type: VOTER last_known_addr { host: "127.9.157.131" port: 38359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:23:39.721000 9032 mvcc.cc:205] Tried to move back new op lower bound from 7317377104727748608 to 7317377104527257600. Current Snapshot: MvccSnapshot[applied={T|T < 7317377104727748608}]
I20260811 17:23:39.729559 8199 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:23:39.729732 8199 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 17:23:39.733052 8199 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 80ef8e421dff44bbbda35c139f8a29f5: Sending DeleteTablet for 3 replicas of tablet 794a9c93dbab4767a1c21fca8bc1f458
I20260811 17:23:39.734091 8423 tablet_service.cc:1514] Processing DeleteTablet for tablet 794a9c93dbab4767a1c21fca8bc1f458 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:23:39 UTC) from {username='slave'} at 127.0.0.1:42112
I20260811 17:23:39.734089 8665 tablet_service.cc:1514] Processing DeleteTablet for tablet 794a9c93dbab4767a1c21fca8bc1f458 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:23:39 UTC) from {username='slave'} at 127.0.0.1:45844
I20260811 17:23:39.734093 8879 tablet_service.cc:1514] Processing DeleteTablet for tablet 794a9c93dbab4767a1c21fca8bc1f458 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 17:23:39 UTC) from {username='slave'} at 127.0.0.1:42430
I20260811 17:23:39.734967 9047 tablet_replica.cc:330] T 794a9c93dbab4767a1c21fca8bc1f458 P 94bb023a08d147348cbbca4c7ae187e6: stopping tablet replica
I20260811 17:23:39.735412 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 8308
I20260811 17:23:39.743726 9047 raft_consensus.cc:2229] T 794a9c93dbab4767a1c21fca8bc1f458 P 94bb023a08d147348cbbca4c7ae187e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 17:23:39.743991 9043 tablet_replica.cc:330] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840: stopping tablet replica
I20260811 17:23:39.744215 9047 raft_consensus.cc:2258] T 794a9c93dbab4767a1c21fca8bc1f458 P 94bb023a08d147348cbbca4c7ae187e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:23:39.744424 9043 raft_consensus.cc:2229] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840 [term 1 LEADER]: Raft consensus shutting down.
I20260811 17:23:39.744848 9043 raft_consensus.cc:2258] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 17:23:39.746163 9047 ts_tablet_manager.cc:1905] T 794a9c93dbab4767a1c21fca8bc1f458 P 94bb023a08d147348cbbca4c7ae187e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:23:39.746544 9043 ts_tablet_manager.cc:1905] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 17:23:39.749176 9047 ts_tablet_manager.cc:1918] T 794a9c93dbab4767a1c21fca8bc1f458 P 94bb023a08d147348cbbca4c7ae187e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:23:39.749289 9047 log.cc:1196] T 794a9c93dbab4767a1c21fca8bc1f458 P 94bb023a08d147348cbbca4c7ae187e6: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-2/wal/wals/794a9c93dbab4767a1c21fca8bc1f458
I20260811 17:23:39.749619 9047 ts_tablet_manager.cc:1939] T 794a9c93dbab4767a1c21fca8bc1f458 P 94bb023a08d147348cbbca4c7ae187e6: Deleting consensus metadata
I20260811 17:23:39.750418 9043 ts_tablet_manager.cc:1918] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 17:23:39.750602 9043 log.cc:1196] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840: Deleting WAL directory at /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786468743878666-9846-0/minicluster-data/ts-1/wal/wals/794a9c93dbab4767a1c21fca8bc1f458
I20260811 17:23:39.751016 9043 ts_tablet_manager.cc:1939] T 794a9c93dbab4767a1c21fca8bc1f458 P 14d440c20aa24e0796bb55a07b4d3840: Deleting consensus metadata
I20260811 17:23:39.751320 8178 catalog_manager.cc:4841] TS 94bb023a08d147348cbbca4c7ae187e6 (127.9.157.131:38359): tablet 794a9c93dbab4767a1c21fca8bc1f458 (table test-table [id=7434468cee6e4ce89fcb2be8521b39c9]) successfully deleted
I20260811 17:23:39.752434 8179 catalog_manager.cc:4841] TS 14d440c20aa24e0796bb55a07b4d3840 (127.9.157.130:35331): tablet 794a9c93dbab4767a1c21fca8bc1f458 (table test-table [id=7434468cee6e4ce89fcb2be8521b39c9]) successfully deleted
W20260811 17:23:39.753239 8180 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.157.129:36835 (error 108)
I20260811 17:23:39.753885 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 8548
I20260811 17:23:39.764500 9007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:39.764091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.129:36835 (local address 127.0.0.1:42126)
0811 17:23:39.764241 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 17:23:39.764358 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.129:36835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 17:23:39.764745 8180 catalog_manager.cc:4584] TS 7c20503843b04322b2a50cc62ee2666f (127.9.157.129:36835): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 794a9c93dbab4767a1c21fca8bc1f458: Network error: Client connection negotiation failed: client connection to 127.9.157.129:36835: connect: Connection refused (error 111)
I20260811 17:23:39.769886 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 8779
I20260811 17:23:39.788566 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 8154
2026-08-11T17:23:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1362 ms)
[----------] 6 tests from EncryptionPolicyTest (12949 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:23:39 dist-test-slave-mjjr krb5kdc[9055](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:23:39 dist-test-slave-mjjr krb5kdc[9055](info): set up 2 sockets
Aug 11 17:23:39 dist-test-slave-mjjr krb5kdc[9055](info): commencing operation
krb5kdc: starting...
W20260811 17:23:41.063668 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:23:41 dist-test-slave-mjjr krb5kdc[9055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:23:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:23:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 17:23:41.199021 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43361
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43361
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:23:41.325204 9079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:41.325585 9079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:41.325678 9079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:41.330487 9079 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:23:41.330612 9079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:41.330657 9079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:41.330703 9079 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:23:41.330757 9079 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:23:41.337247 9079 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43361
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:41.338601 9079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:41.339851 9079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:41.349852 9090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:41.349856 9087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:41.349856 9086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:41.350244 9079 server_base.cc:1034] running on GCE node
I20260811 17:23:41.350940 9079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:41.352283 9079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:41.353533 9079 hybrid_clock.cc:648] HybridClock initialized: now 1786469021353477 us; error 73 us; skew 500 ppm
Aug 11 17:23:41 dist-test-slave-mjjr krb5kdc[9055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469021, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:23:41.360375 9079 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 17:23:41.361438 9079 webserver.cc:460] Webserver started at http://127.0.0.1:39091/ using document root <none> and password file <none>
I20260811 17:23:41.362023 9079 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:41.362089 9079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:41.362283 9079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:41.364630 9079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "9df55e7e2dfb4a67844294f721cae51a"
format_stamp: "Formatted at 2026-08-11 17:23:41 on dist-test-slave-mjjr"
server_key: "9c66ef2bce5f585ad9c4dda133e6de5c"
server_key_iv: "f6cf61508db4a67bda49fae4c0d418fd"
server_key_version: "encryptionkey@0"
I20260811 17:23:41.365208 9079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "9df55e7e2dfb4a67844294f721cae51a"
format_stamp: "Formatted at 2026-08-11 17:23:41 on dist-test-slave-mjjr"
server_key: "9c66ef2bce5f585ad9c4dda133e6de5c"
server_key_iv: "f6cf61508db4a67bda49fae4c0d418fd"
server_key_version: "encryptionkey@0"
I20260811 17:23:41.369463 9079 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:23:41.372150 9100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:41.373617 9079 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:23:41.373780 9079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "9df55e7e2dfb4a67844294f721cae51a"
format_stamp: "Formatted at 2026-08-11 17:23:41 on dist-test-slave-mjjr"
server_key: "9c66ef2bce5f585ad9c4dda133e6de5c"
server_key_iv: "f6cf61508db4a67bda49fae4c0d418fd"
server_key_version: "encryptionkey@0"
I20260811 17:23:41.373917 9079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:41.388597 9079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:41.392369 9079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:41.392618 9079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:41.400647 9079 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43361
I20260811 17:23:41.400657 9190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43361 every 8 connection(s)
I20260811 17:23:41.401738 9079 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:23:41.404902 9192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:41.408023 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 9079
I20260811 17:23:41.408193 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:23:41.408533 9846 external_mini_cluster.cc:1428] Setting key b64cc501e4757270f3eef78b19ccf476
I20260811 17:23:41.411790 9192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9df55e7e2dfb4a67844294f721cae51a: Bootstrap starting.
I20260811 17:23:41.414628 9192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9df55e7e2dfb4a67844294f721cae51a: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:41.415630 9192 log.cc:824] T 00000000000000000000000000000000 P 9df55e7e2dfb4a67844294f721cae51a: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:41.418537 9192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9df55e7e2dfb4a67844294f721cae51a: No bootstrap required, opened a new log
Aug 11 17:23:41 dist-test-slave-mjjr krb5kdc[9055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 17:23:41.422020 9192 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9df55e7e2dfb4a67844294f721cae51a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9df55e7e2dfb4a67844294f721cae51a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43361 } }
I20260811 17:23:41.422271 9192 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9df55e7e2dfb4a67844294f721cae51a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:41.422358 9192 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9df55e7e2dfb4a67844294f721cae51a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9df55e7e2dfb4a67844294f721cae51a, State: Initialized, Role: FOLLOWER
I20260811 17:23:41.422820 9192 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9df55e7e2dfb4a67844294f721cae51a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9df55e7e2dfb4a67844294f721cae51a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43361 } }
I20260811 17:23:41.422950 9192 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9df55e7e2dfb4a67844294f721cae51a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:23:41.423038 9192 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9df55e7e2dfb4a67844294f721cae51a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:23:41.423147 9192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9df55e7e2dfb4a67844294f721cae51a [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:41.424466 9192 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9df55e7e2dfb4a67844294f721cae51a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9df55e7e2dfb4a67844294f721cae51a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43361 } }
I20260811 17:23:41.424860 9192 leader_election.cc:302] T 00000000000000000000000000000000 P 9df55e7e2dfb4a67844294f721cae51a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9df55e7e2dfb4a67844294f721cae51a; no voters:
I20260811 17:23:41.425261 9192 leader_election.cc:288] T 00000000000000000000000000000000 P 9df55e7e2dfb4a67844294f721cae51a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:23:41.425359 9198 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9df55e7e2dfb4a67844294f721cae51a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:23:41.425572 9198 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9df55e7e2dfb4a67844294f721cae51a [term 1 LEADER]: Becoming Leader. State: Replica: 9df55e7e2dfb4a67844294f721cae51a, State: Running, Role: LEADER
I20260811 17:23:41.426111 9198 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9df55e7e2dfb4a67844294f721cae51a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9df55e7e2dfb4a67844294f721cae51a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43361 } }
I20260811 17:23:41.426651 9196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:41.410203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54909 (local address 127.0.0.1:43361)
0811 17:23:41.410744 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:41.410763 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:41.410803 (+ 40us) server_negotiation.cc:407] Connection header received
0811 17:23:41.411509 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:41.411528 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:41.412043 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:41.412364 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:41.413347 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:41.414491 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:41.415824 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:41.416575 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:41.416770 (+ 195us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:41.420053 (+ 3283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:41.420059 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:41.420075 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:23:41.420107 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:41.422852 (+ 2745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:41.423713 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:41.423720 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:41.423727 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:41.423864 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:41.424275 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:41.424278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:41.424279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:41.424792 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:41.425080 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:41.425469 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 17:23:41.425735 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":186,"threads_started":1}
I20260811 17:23:41.426769 9192 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9df55e7e2dfb4a67844294f721cae51a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:23:41.427431 9198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9df55e7e2dfb4a67844294f721cae51a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9df55e7e2dfb4a67844294f721cae51a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9df55e7e2dfb4a67844294f721cae51a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43361 } } }
I20260811 17:23:41.427556 9198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9df55e7e2dfb4a67844294f721cae51a [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:41.427804 9198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9df55e7e2dfb4a67844294f721cae51a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9df55e7e2dfb4a67844294f721cae51a. Latest consensus state: current_term: 1 leader_uuid: "9df55e7e2dfb4a67844294f721cae51a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9df55e7e2dfb4a67844294f721cae51a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43361 } } }
I20260811 17:23:41.427922 9198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9df55e7e2dfb4a67844294f721cae51a [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:41.428459 9215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:23:41.430258 9846 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:23:41.432123 9215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:23:41.438999 9215 catalog_manager.cc:1347] Generated new cluster ID: 9bcf7cd344c749c386d3197e2cacb845
I20260811 17:23:41.439091 9215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:23:41.443383 9196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:41.432654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50813 (local address 127.0.0.1:43361)
0811 17:23:41.432772 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:41.432777 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:41.434645 (+ 1868us) server_negotiation.cc:407] Connection header received
0811 17:23:41.434777 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:41.434780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:41.434834 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:41.434924 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:41.435808 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:41.436496 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:41.437501 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:41.437750 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:41.437866 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:41.439688 (+ 1822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:41.439692 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:41.439695 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:23:41.439724 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:41.441415 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:41.441961 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:41.441965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:41.441967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:41.442052 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:41.442393 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:41.442397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:41.442398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:41.442599 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:41.442720 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:41.443117 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 17:23:41.443218 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 17:23:41.469693 9215 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:23:41.470875 9215 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:23:41.476464 9215 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9df55e7e2dfb4a67844294f721cae51a: Generated new TSK 0
I20260811 17:23:41.477175 9215 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:23:41.482374 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 9079
W20260811 17:23:41.501284 9049 connection.cc:535] client connection to 127.0.0.1:43361 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43361): Connection reset by peer (error 104)
2026-08-11T17:23:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1696 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:23:41 dist-test-slave-mjjr krb5kdc[9246](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:23:41 dist-test-slave-mjjr krb5kdc[9246](info): set up 2 sockets
Aug 11 17:23:41 dist-test-slave-mjjr krb5kdc[9246](info): commencing operation
krb5kdc: starting...
W20260811 17:23:42.756830 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:23:42 dist-test-slave-mjjr krb5kdc[9246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:23:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:23:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 17:23:42.899847 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34747
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:23:43.026073 9276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:43.026484 9276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:43.026603 9276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:43.031072 9276 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:23:43.031205 9276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:43.031248 9276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:43.031296 9276 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:23:43.031337 9276 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:23:43.037111 9276 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34747
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:43.038468 9276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:43.039721 9276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:23:43.049116 9276 server_base.cc:1034] running on GCE node
W20260811 17:23:43.049065 9285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:43.049149 9284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:43.049065 9288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:43.049980 9276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:43.051234 9276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:43.052443 9276 hybrid_clock.cc:648] HybridClock initialized: now 1786469023052421 us; error 35 us; skew 500 ppm
Aug 11 17:23:43 dist-test-slave-mjjr krb5kdc[9246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469023, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:23:43.055526 9276 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 17:23:43.056564 9276 webserver.cc:460] Webserver started at http://127.0.0.1:38613/ using document root <none> and password file <none>
I20260811 17:23:43.057161 9276 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:43.057258 9276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:43.057509 9276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:43.059810 9276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "0be1fc1081354123a43c4fbfc0afa42a"
format_stamp: "Formatted at 2026-08-11 17:23:43 on dist-test-slave-mjjr"
server_key: "86c4e8f1e2f2cdd2e4f4de14ef3db834"
server_key_iv: "0074629e2cd4700138dc5cc92067e87c"
server_key_version: "encryptionkey@0"
I20260811 17:23:43.060417 9276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "0be1fc1081354123a43c4fbfc0afa42a"
format_stamp: "Formatted at 2026-08-11 17:23:43 on dist-test-slave-mjjr"
server_key: "86c4e8f1e2f2cdd2e4f4de14ef3db834"
server_key_iv: "0074629e2cd4700138dc5cc92067e87c"
server_key_version: "encryptionkey@0"
I20260811 17:23:43.064471 9276 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 17:23:43.067087 9301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:43.068488 9276 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 17:23:43.068643 9276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "0be1fc1081354123a43c4fbfc0afa42a"
format_stamp: "Formatted at 2026-08-11 17:23:43 on dist-test-slave-mjjr"
server_key: "86c4e8f1e2f2cdd2e4f4de14ef3db834"
server_key_iv: "0074629e2cd4700138dc5cc92067e87c"
server_key_version: "encryptionkey@0"
I20260811 17:23:43.068773 9276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:43.085319 9276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:43.089129 9276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:43.089361 9276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:43.097714 9276 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34747
I20260811 17:23:43.097743 9393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34747 every 8 connection(s)
I20260811 17:23:43.098872 9276 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:23:43.101878 9399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:43.107990 9399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0be1fc1081354123a43c4fbfc0afa42a: Bootstrap starting.
I20260811 17:23:43.109094 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 9276
I20260811 17:23:43.109266 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:23:43.109596 9846 external_mini_cluster.cc:1428] Setting key aceec2dbc8d8e7f8cedef43ec517921e
I20260811 17:23:43.111027 9399 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0be1fc1081354123a43c4fbfc0afa42a: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:43.111924 9399 log.cc:824] T 00000000000000000000000000000000 P 0be1fc1081354123a43c4fbfc0afa42a: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:43.114603 9399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0be1fc1081354123a43c4fbfc0afa42a: No bootstrap required, opened a new log
I20260811 17:23:43.119866 9399 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0be1fc1081354123a43c4fbfc0afa42a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0be1fc1081354123a43c4fbfc0afa42a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34747 } }
I20260811 17:23:43.120173 9399 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0be1fc1081354123a43c4fbfc0afa42a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:43.120257 9399 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0be1fc1081354123a43c4fbfc0afa42a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0be1fc1081354123a43c4fbfc0afa42a, State: Initialized, Role: FOLLOWER
I20260811 17:23:43.120625 9405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:43.110880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58331 (local address 127.0.0.1:34747)
0811 17:23:43.111434 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:43.111453 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:43.111493 (+ 40us) server_negotiation.cc:407] Connection header received
0811 17:23:43.112381 (+ 888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:43.112401 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:43.112852 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:43.113275 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:43.113997 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:43.115088 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:43.116463 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:43.116994 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:43.117139 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:43.118192 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:43.118199 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:43.118216 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:43.118745 (+ 529us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:43.118827 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:43.118863 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:43.119001 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:43.119329 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 17:23:43.119514 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":167,"threads_started":1}
I20260811 17:23:43.120846 9399 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0be1fc1081354123a43c4fbfc0afa42a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0be1fc1081354123a43c4fbfc0afa42a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34747 } }
I20260811 17:23:43.121028 9399 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0be1fc1081354123a43c4fbfc0afa42a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:23:43.121104 9399 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0be1fc1081354123a43c4fbfc0afa42a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:23:43.121206 9399 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0be1fc1081354123a43c4fbfc0afa42a [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:43.122427 9399 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0be1fc1081354123a43c4fbfc0afa42a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0be1fc1081354123a43c4fbfc0afa42a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34747 } }
I20260811 17:23:43.122819 9399 leader_election.cc:302] T 00000000000000000000000000000000 P 0be1fc1081354123a43c4fbfc0afa42a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0be1fc1081354123a43c4fbfc0afa42a; no voters:
I20260811 17:23:43.123230 9399 leader_election.cc:288] T 00000000000000000000000000000000 P 0be1fc1081354123a43c4fbfc0afa42a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:23:43.123410 9407 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0be1fc1081354123a43c4fbfc0afa42a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:23:43.123710 9407 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0be1fc1081354123a43c4fbfc0afa42a [term 1 LEADER]: Becoming Leader. State: Replica: 0be1fc1081354123a43c4fbfc0afa42a, State: Running, Role: LEADER
I20260811 17:23:43.124287 9407 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0be1fc1081354123a43c4fbfc0afa42a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0be1fc1081354123a43c4fbfc0afa42a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34747 } }
I20260811 17:23:43.124553 9846 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:23:43.124854 9399 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0be1fc1081354123a43c4fbfc0afa42a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:23:43.125718 9407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0be1fc1081354123a43c4fbfc0afa42a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0be1fc1081354123a43c4fbfc0afa42a. Latest consensus state: current_term: 1 leader_uuid: "0be1fc1081354123a43c4fbfc0afa42a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0be1fc1081354123a43c4fbfc0afa42a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34747 } } }
I20260811 17:23:43.125864 9407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0be1fc1081354123a43c4fbfc0afa42a [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:43.126313 9409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0be1fc1081354123a43c4fbfc0afa42a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0be1fc1081354123a43c4fbfc0afa42a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0be1fc1081354123a43c4fbfc0afa42a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34747 } } }
I20260811 17:23:43.126428 9409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0be1fc1081354123a43c4fbfc0afa42a [sys.catalog]: This master's current role is: LEADER
W20260811 17:23:43.130597 9441 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0be1fc1081354123a43c4fbfc0afa42a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:23:43.130728 9441 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:23:43.130820 9426 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:23:43.131860 9426 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:23:43.132934 9405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:43.127177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50407 (local address 127.0.0.1:34747)
0811 17:23:43.127297 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:43.127304 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:43.128052 (+ 748us) server_negotiation.cc:407] Connection header received
0811 17:23:43.128306 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:43.128310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:43.128363 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:43.128447 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:43.128789 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:43.129571 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:43.130799 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:43.131142 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:43.131264 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:43.132378 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:43.132383 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:43.132386 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:43.132403 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:43.132434 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:43.132449 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:43.132565 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:43.132747 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 17:23:43.132791 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 17:23:43.138526 9426 catalog_manager.cc:1347] Generated new cluster ID: c9aac6fe93184027861d569c6895cd7e
I20260811 17:23:43.138622 9426 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:23:43.149432 9426 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:23:43.150547 9426 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:23:43.162147 9426 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0be1fc1081354123a43c4fbfc0afa42a: Generated new TSK 0
I20260811 17:23:43.163151 9426 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:23:43.171967 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 9276
W20260811 17:23:43.185755 9244 connection.cc:535] client connection to 127.0.0.1:34747 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34747): Connection reset by peer (error 104)
2026-08-11T17:23:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1684 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:23:43 dist-test-slave-mjjr krb5kdc[9454](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:23:43 dist-test-slave-mjjr krb5kdc[9454](info): set up 2 sockets
Aug 11 17:23:43 dist-test-slave-mjjr krb5kdc[9454](info): commencing operation
krb5kdc: starting...
W20260811 17:23:44.448382 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:23:44 dist-test-slave-mjjr krb5kdc[9454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:23:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:23:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 17:23:44.590905 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36007
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36007
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:23:44.711891 9473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:44.712200 9473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:44.712253 9473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:44.716344 9473 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:23:44.716419 9473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:44.716439 9473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:44.716457 9473 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:23:44.716473 9473 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:23:44.722019 9473 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36007
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:44.723169 9473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:44.724228 9473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:44.733352 9481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:44.733352 9484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:44.733356 9486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:44.733448 9473 server_base.cc:1034] running on GCE node
I20260811 17:23:44.734261 9473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:44.735525 9473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:44.736714 9473 hybrid_clock.cc:648] HybridClock initialized: now 1786469024736702 us; error 28 us; skew 500 ppm
Aug 11 17:23:44 dist-test-slave-mjjr krb5kdc[9454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469024, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:23:44.739580 9473 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 17:23:44.740571 9473 webserver.cc:460] Webserver started at http://127.0.0.1:46713/ using document root <none> and password file <none>
I20260811 17:23:44.741195 9473 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:44.741291 9473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:44.741546 9473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:44.744204 9473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "5092d3501413489ab63a851cf1592f3e"
format_stamp: "Formatted at 2026-08-11 17:23:44 on dist-test-slave-mjjr"
server_key: "8dd8c3cada1aaa74b4818f00f7232fda"
server_key_iv: "accde027bab11fd870e493e9dba302af"
server_key_version: "encryptionkey@0"
I20260811 17:23:44.744892 9473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "5092d3501413489ab63a851cf1592f3e"
format_stamp: "Formatted at 2026-08-11 17:23:44 on dist-test-slave-mjjr"
server_key: "8dd8c3cada1aaa74b4818f00f7232fda"
server_key_iv: "accde027bab11fd870e493e9dba302af"
server_key_version: "encryptionkey@0"
I20260811 17:23:44.749202 9473 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:23:44.752020 9494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:44.753288 9473 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 17:23:44.753443 9473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "5092d3501413489ab63a851cf1592f3e"
format_stamp: "Formatted at 2026-08-11 17:23:44 on dist-test-slave-mjjr"
server_key: "8dd8c3cada1aaa74b4818f00f7232fda"
server_key_iv: "accde027bab11fd870e493e9dba302af"
server_key_version: "encryptionkey@0"
I20260811 17:23:44.753575 9473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:44.758383 9473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:44.761647 9473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:44.761852 9473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:44.769512 9473 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36007
I20260811 17:23:44.769562 9571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36007 every 8 connection(s)
I20260811 17:23:44.770967 9473 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:23:44.775223 9573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:44.779299 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 9473
I20260811 17:23:44.779475 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:23:44.779884 9846 external_mini_cluster.cc:1428] Setting key a7f2e9e0f030805e9eaba52add0905f0
I20260811 17:23:44.782689 9573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5092d3501413489ab63a851cf1592f3e: Bootstrap starting.
I20260811 17:23:44.785343 9573 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5092d3501413489ab63a851cf1592f3e: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:44.785867 9586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:44.781419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48539 (local address 127.0.0.1:36007)
0811 17:23:44.781928 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:44.781952 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:44.782148 (+ 196us) server_negotiation.cc:407] Connection header received
0811 17:23:44.783050 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:44.783068 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:44.783413 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:44.783807 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:44.784563 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:44.784568 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:44.784581 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:44.784592 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:44.784643 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:44.784661 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:44.784794 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:44.784896 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 17:23:44.785075 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":146,"threads_started":1}
I20260811 17:23:44.786262 9573 log.cc:824] T 00000000000000000000000000000000 P 5092d3501413489ab63a851cf1592f3e: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:44.788682 9573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5092d3501413489ab63a851cf1592f3e: No bootstrap required, opened a new log
I20260811 17:23:44.789090 9846 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:23:44.793071 9586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:44.791220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53617 (local address 127.0.0.1:36007)
0811 17:23:44.791362 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:44.791369 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:44.791600 (+ 231us) server_negotiation.cc:407] Connection header received
0811 17:23:44.791757 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:44.791761 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:44.791808 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:44.791925 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:44.792700 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:44.792704 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:44.792706 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:44.792719 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:44.792747 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:44.792759 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:44.792855 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:44.792913 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 17:23:44.792951 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 17:23:44.793104 9573 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5092d3501413489ab63a851cf1592f3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5092d3501413489ab63a851cf1592f3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36007 } }
I20260811 17:23:44.793411 9573 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5092d3501413489ab63a851cf1592f3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:44.793471 9573 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5092d3501413489ab63a851cf1592f3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5092d3501413489ab63a851cf1592f3e, State: Initialized, Role: FOLLOWER
I20260811 17:23:44.793982 9573 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5092d3501413489ab63a851cf1592f3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5092d3501413489ab63a851cf1592f3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36007 } }
I20260811 17:23:44.794148 9573 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5092d3501413489ab63a851cf1592f3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:23:44.794222 9573 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5092d3501413489ab63a851cf1592f3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:23:44.794322 9573 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5092d3501413489ab63a851cf1592f3e [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:44.795323 9573 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5092d3501413489ab63a851cf1592f3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5092d3501413489ab63a851cf1592f3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36007 } }
I20260811 17:23:44.795733 9573 leader_election.cc:302] T 00000000000000000000000000000000 P 5092d3501413489ab63a851cf1592f3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5092d3501413489ab63a851cf1592f3e; no voters:
I20260811 17:23:44.796079 9573 leader_election.cc:288] T 00000000000000000000000000000000 P 5092d3501413489ab63a851cf1592f3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:23:44.796221 9596 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5092d3501413489ab63a851cf1592f3e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:23:44.796512 9596 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5092d3501413489ab63a851cf1592f3e [term 1 LEADER]: Becoming Leader. State: Replica: 5092d3501413489ab63a851cf1592f3e, State: Running, Role: LEADER
I20260811 17:23:44.796835 9596 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5092d3501413489ab63a851cf1592f3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5092d3501413489ab63a851cf1592f3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36007 } }
I20260811 17:23:44.797413 9573 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5092d3501413489ab63a851cf1592f3e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:23:44.798559 9597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5092d3501413489ab63a851cf1592f3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5092d3501413489ab63a851cf1592f3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5092d3501413489ab63a851cf1592f3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36007 } } }
I20260811 17:23:44.798712 9597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5092d3501413489ab63a851cf1592f3e [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:44.799157 9607 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:23:44.799105 9598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5092d3501413489ab63a851cf1592f3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5092d3501413489ab63a851cf1592f3e. Latest consensus state: current_term: 1 leader_uuid: "5092d3501413489ab63a851cf1592f3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5092d3501413489ab63a851cf1592f3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36007 } } }
I20260811 17:23:44.799278 9598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5092d3501413489ab63a851cf1592f3e [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:44.802196 9607 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:23:44.808876 9607 catalog_manager.cc:1347] Generated new cluster ID: 4ef225ade1854d01bdb4ae8ff0425349
I20260811 17:23:44.809018 9607 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:23:44.821592 9607 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:23:44.821990 9607 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:23:44.828279 9607 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5092d3501413489ab63a851cf1592f3e: Generated new TSK 0
I20260811 17:23:44.829232 9607 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:23:44.840847 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 9473
2026-08-11T17:23:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1671 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:23:44 dist-test-slave-mjjr krb5kdc[9626](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:23:44 dist-test-slave-mjjr krb5kdc[9626](info): set up 2 sockets
Aug 11 17:23:44 dist-test-slave-mjjr krb5kdc[9626](info): commencing operation
krb5kdc: starting...
W20260811 17:23:46.116628 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:23:46 dist-test-slave-mjjr krb5kdc[9626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:23:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:23:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 17:23:46.255882 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36377
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:23:46.386812 9646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:46.387389 9646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:46.387529 9646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:46.394948 9646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:23:46.395032 9646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:46.395053 9646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:46.395072 9646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:23:46.395088 9646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:23:46.404671 9646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:46.406177 9646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:46.407629 9646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:46.418072 9653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:46.418072 9654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:46.418613 9656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:46.421252 9646 server_base.cc:1034] running on GCE node
I20260811 17:23:46.422093 9646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:46.423816 9646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:46.425026 9646 hybrid_clock.cc:648] HybridClock initialized: now 1786469026424990 us; error 56 us; skew 500 ppm
Aug 11 17:23:46 dist-test-slave-mjjr krb5kdc[9626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469026, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:23:46.428861 9646 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 17:23:46.430126 9646 webserver.cc:460] Webserver started at http://127.0.0.1:34725/ using document root <none> and password file <none>
I20260811 17:23:46.430922 9646 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:46.431056 9646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:46.431345 9646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:46.434626 9646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "5662a8c3e2da4926a4910cd29dcd48d9"
format_stamp: "Formatted at 2026-08-11 17:23:46 on dist-test-slave-mjjr"
server_key: "ef5c05928f7cc6eb4b94b06fe207e1c1"
server_key_iv: "cf93db123aa2fa5bc0153fb41aa96fa8"
server_key_version: "encryptionkey@0"
I20260811 17:23:46.435377 9646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "5662a8c3e2da4926a4910cd29dcd48d9"
format_stamp: "Formatted at 2026-08-11 17:23:46 on dist-test-slave-mjjr"
server_key: "ef5c05928f7cc6eb4b94b06fe207e1c1"
server_key_iv: "cf93db123aa2fa5bc0153fb41aa96fa8"
server_key_version: "encryptionkey@0"
I20260811 17:23:46.439886 9646 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 17:23:46.442468 9668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:46.443953 9646 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:23:46.444096 9646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "5662a8c3e2da4926a4910cd29dcd48d9"
format_stamp: "Formatted at 2026-08-11 17:23:46 on dist-test-slave-mjjr"
server_key: "ef5c05928f7cc6eb4b94b06fe207e1c1"
server_key_iv: "cf93db123aa2fa5bc0153fb41aa96fa8"
server_key_version: "encryptionkey@0"
I20260811 17:23:46.444223 9646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:46.467710 9646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:46.471418 9646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:46.471643 9646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:46.479758 9646 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36377
I20260811 17:23:46.479914 9753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36377 every 8 connection(s)
I20260811 17:23:46.480860 9646 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:23:46.483916 9754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:46.485109 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 9646
I20260811 17:23:46.485270 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:23:46.485615 9846 external_mini_cluster.cc:1428] Setting key c5762fb8a556ecc161be9a45c82dcbeb
I20260811 17:23:46.490723 9754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5662a8c3e2da4926a4910cd29dcd48d9: Bootstrap starting.
I20260811 17:23:46.493527 9754 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5662a8c3e2da4926a4910cd29dcd48d9: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:46.494354 9754 log.cc:824] T 00000000000000000000000000000000 P 5662a8c3e2da4926a4910cd29dcd48d9: Log is configured to *not* fsync() on all Append() calls
Aug 11 17:23:46 dist-test-slave-mjjr krb5kdc[9626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 17:23:46.497013 9754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5662a8c3e2da4926a4910cd29dcd48d9: No bootstrap required, opened a new log
I20260811 17:23:46.500859 9758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:46.487214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51179 (local address 127.0.0.1:36377)
0811 17:23:46.487754 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:46.487771 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:46.487812 (+ 41us) server_negotiation.cc:407] Connection header received
0811 17:23:46.488471 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:46.488489 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:46.488843 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:46.489199 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:46.490081 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:46.491160 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:46.492097 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:46.492671 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:46.492851 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:46.495786 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:46.495792 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:46.495808 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:23:46.495835 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:46.497659 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:46.498218 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:46.498223 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:46.498228 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:46.498345 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:46.498665 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:46.498668 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:46.498670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:46.499113 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:46.499271 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:46.499700 (+ 429us) server_negotiation.cc:299] Negotiation successful
0811 17:23:46.500056 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":140,"threads_started":1}
I20260811 17:23:46.501524 9754 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5662a8c3e2da4926a4910cd29dcd48d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5662a8c3e2da4926a4910cd29dcd48d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36377 } }
I20260811 17:23:46.501819 9754 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5662a8c3e2da4926a4910cd29dcd48d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:46.501881 9754 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5662a8c3e2da4926a4910cd29dcd48d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5662a8c3e2da4926a4910cd29dcd48d9, State: Initialized, Role: FOLLOWER
I20260811 17:23:46.502398 9754 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5662a8c3e2da4926a4910cd29dcd48d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5662a8c3e2da4926a4910cd29dcd48d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36377 } }
I20260811 17:23:46.502568 9754 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5662a8c3e2da4926a4910cd29dcd48d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:23:46.502636 9754 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5662a8c3e2da4926a4910cd29dcd48d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:23:46.502737 9754 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5662a8c3e2da4926a4910cd29dcd48d9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:46.504077 9754 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5662a8c3e2da4926a4910cd29dcd48d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5662a8c3e2da4926a4910cd29dcd48d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36377 } }
I20260811 17:23:46.504251 9846 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:23:46.504511 9754 leader_election.cc:302] T 00000000000000000000000000000000 P 5662a8c3e2da4926a4910cd29dcd48d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5662a8c3e2da4926a4910cd29dcd48d9; no voters:
I20260811 17:23:46.504956 9754 leader_election.cc:288] T 00000000000000000000000000000000 P 5662a8c3e2da4926a4910cd29dcd48d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:23:46.505056 9761 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5662a8c3e2da4926a4910cd29dcd48d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:23:46.505244 9761 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5662a8c3e2da4926a4910cd29dcd48d9 [term 1 LEADER]: Becoming Leader. State: Replica: 5662a8c3e2da4926a4910cd29dcd48d9, State: Running, Role: LEADER
I20260811 17:23:46.505614 9761 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5662a8c3e2da4926a4910cd29dcd48d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5662a8c3e2da4926a4910cd29dcd48d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36377 } }
I20260811 17:23:46.506379 9754 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5662a8c3e2da4926a4910cd29dcd48d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:23:46.506937 9761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5662a8c3e2da4926a4910cd29dcd48d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5662a8c3e2da4926a4910cd29dcd48d9. Latest consensus state: current_term: 1 leader_uuid: "5662a8c3e2da4926a4910cd29dcd48d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5662a8c3e2da4926a4910cd29dcd48d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36377 } } }
I20260811 17:23:46.507120 9761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5662a8c3e2da4926a4910cd29dcd48d9 [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:46.507390 9764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5662a8c3e2da4926a4910cd29dcd48d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5662a8c3e2da4926a4910cd29dcd48d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5662a8c3e2da4926a4910cd29dcd48d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36377 } } }
I20260811 17:23:46.507501 9764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5662a8c3e2da4926a4910cd29dcd48d9 [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:46.508980 9776 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:23:46.512259 9776 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:23:46.518126 9758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:46.508343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52599 (local address 127.0.0.1:36377)
0811 17:23:46.508469 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:46.508474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:46.508848 (+ 374us) server_negotiation.cc:407] Connection header received
0811 17:23:46.509054 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:46.509057 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:46.509110 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:46.509188 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:46.510101 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:46.510857 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:46.512036 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:46.512379 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:46.512489 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:46.514190 (+ 1701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:46.514195 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:46.514198 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:23:46.514228 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:46.516188 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:46.516717 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:46.516721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:46.516723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:46.516820 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:46.517139 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:46.517143 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:46.517144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:46.517402 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:46.517547 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:46.517833 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 17:23:46.517956 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 17:23:46.519261 9776 catalog_manager.cc:1347] Generated new cluster ID: 1d85123ea1f74959a81ac9a1bb3a4e12
I20260811 17:23:46.519383 9776 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:23:46.547837 9776 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:23:46.549357 9776 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:23:46.560156 9776 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5662a8c3e2da4926a4910cd29dcd48d9: Generated new TSK 0
I20260811 17:23:46.561187 9776 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:23:46.568643 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 9646
2026-08-11T17:23:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1726 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:23:46 dist-test-slave-mjjr krb5kdc[9803](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:23:46 dist-test-slave-mjjr krb5kdc[9803](info): set up 2 sockets
Aug 11 17:23:46 dist-test-slave-mjjr krb5kdc[9803](info): commencing operation
krb5kdc: starting...
W20260811 17:23:47.842970 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:23:47 dist-test-slave-mjjr krb5kdc[9803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:23:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:23:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 17:23:47.987452 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39781
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:23:48.108338 9835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:48.108712 9835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:48.108763 9835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:48.113191 9835 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:23:48.113300 9835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:48.113323 9835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:48.113343 9835 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:23:48.113363 9835 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:23:48.119408 9835 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39781
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:48.120867 9835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:48.122386 9835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:48.133116 9842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:48.133118 9844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:48.133118 9856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:48.133183 9835 server_base.cc:1034] running on GCE node
I20260811 17:23:48.134315 9835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:48.135629 9835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:48.136910 9835 hybrid_clock.cc:648] HybridClock initialized: now 1786469028136869 us; error 52 us; skew 500 ppm
Aug 11 17:23:48 dist-test-slave-mjjr krb5kdc[9803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469028, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:23:48.140432 9835 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 17:23:48.141544 9835 webserver.cc:460] Webserver started at http://127.0.0.1:35009/ using document root <none> and password file <none>
I20260811 17:23:48.142234 9835 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:48.142309 9835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:48.142504 9835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:48.145380 9835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "c1e0f5a979494fb2be312a2f0470141a"
format_stamp: "Formatted at 2026-08-11 17:23:48 on dist-test-slave-mjjr"
server_key: "25c1bdffa196abdaf490f575ecb0934e"
server_key_iv: "a1b2b536e26911f22ee7f8ca5aac78b0"
server_key_version: "encryptionkey@0"
I20260811 17:23:48.146042 9835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "c1e0f5a979494fb2be312a2f0470141a"
format_stamp: "Formatted at 2026-08-11 17:23:48 on dist-test-slave-mjjr"
server_key: "25c1bdffa196abdaf490f575ecb0934e"
server_key_iv: "a1b2b536e26911f22ee7f8ca5aac78b0"
server_key_version: "encryptionkey@0"
I20260811 17:23:48.150985 9835 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 17:23:48.154150 9865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:48.156209 9835 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:23:48.156385 9835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "c1e0f5a979494fb2be312a2f0470141a"
format_stamp: "Formatted at 2026-08-11 17:23:48 on dist-test-slave-mjjr"
server_key: "25c1bdffa196abdaf490f575ecb0934e"
server_key_iv: "a1b2b536e26911f22ee7f8ca5aac78b0"
server_key_version: "encryptionkey@0"
I20260811 17:23:48.156514 9835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:48.168813 9835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:48.172797 9835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:48.173020 9835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:48.182642 9835 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39781
I20260811 17:23:48.182739 9942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39781 every 8 connection(s)
I20260811 17:23:48.184321 9835 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:23:48.186266 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 9835
I20260811 17:23:48.186456 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:23:48.186793 9846 external_mini_cluster.cc:1428] Setting key 0feb97d58bbc81f0debadf5fc69ab964
I20260811 17:23:48.188542 9944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:48.196182 9944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1e0f5a979494fb2be312a2f0470141a: Bootstrap starting.
I20260811 17:23:48.197734 9950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:48.188453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50871 (local address 127.0.0.1:39781)
0811 17:23:48.189140 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:48.189168 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:48.189212 (+ 44us) server_negotiation.cc:407] Connection header received
0811 17:23:48.189915 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:48.189933 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:48.190314 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:48.190647 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:48.191146 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:48.192346 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:48.193645 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:48.194186 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:48.194329 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:48.195473 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:48.195479 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:48.195496 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:48.196101 (+ 605us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:48.196187 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:48.196216 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:48.196318 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:48.196587 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 17:23:48.196901 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":242,"threads_started":1}
I20260811 17:23:48.199328 9944 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1e0f5a979494fb2be312a2f0470141a: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:48.200423 9944 log.cc:824] T 00000000000000000000000000000000 P c1e0f5a979494fb2be312a2f0470141a: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:48.201566 9846 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:23:48.202996 9944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1e0f5a979494fb2be312a2f0470141a: No bootstrap required, opened a new log
I20260811 17:23:48.208539 9944 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1e0f5a979494fb2be312a2f0470141a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e0f5a979494fb2be312a2f0470141a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39781 } }
I20260811 17:23:48.208973 9944 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1e0f5a979494fb2be312a2f0470141a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:48.209097 9944 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1e0f5a979494fb2be312a2f0470141a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e0f5a979494fb2be312a2f0470141a, State: Initialized, Role: FOLLOWER
I20260811 17:23:48.209318 9950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:48.204206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33021 (local address 127.0.0.1:39781)
0811 17:23:48.204362 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:48.204367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:48.204542 (+ 175us) server_negotiation.cc:407] Connection header received
0811 17:23:48.204686 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:48.204689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:48.204733 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:48.204830 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:48.205221 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:48.205751 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:48.207131 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:48.207397 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:48.207500 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:48.208624 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:48.208628 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:48.208630 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:48.208654 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:48.208687 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:48.208699 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:48.208768 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:48.209037 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 17:23:48.209116 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 17:23:48.209719 9944 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1e0f5a979494fb2be312a2f0470141a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e0f5a979494fb2be312a2f0470141a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39781 } }
I20260811 17:23:48.209918 9944 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1e0f5a979494fb2be312a2f0470141a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:23:48.210021 9944 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1e0f5a979494fb2be312a2f0470141a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:23:48.210163 9944 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1e0f5a979494fb2be312a2f0470141a [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:48.211550 9944 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1e0f5a979494fb2be312a2f0470141a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e0f5a979494fb2be312a2f0470141a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39781 } }
I20260811 17:23:48.212096 9944 leader_election.cc:302] T 00000000000000000000000000000000 P c1e0f5a979494fb2be312a2f0470141a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1e0f5a979494fb2be312a2f0470141a; no voters:
I20260811 17:23:48.212555 9944 leader_election.cc:288] T 00000000000000000000000000000000 P c1e0f5a979494fb2be312a2f0470141a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:23:48.212832 9961 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1e0f5a979494fb2be312a2f0470141a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:23:48.213145 9961 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1e0f5a979494fb2be312a2f0470141a [term 1 LEADER]: Becoming Leader. State: Replica: c1e0f5a979494fb2be312a2f0470141a, State: Running, Role: LEADER
I20260811 17:23:48.213578 9961 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1e0f5a979494fb2be312a2f0470141a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e0f5a979494fb2be312a2f0470141a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39781 } }
I20260811 17:23:48.214252 9944 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1e0f5a979494fb2be312a2f0470141a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:23:48.216226 9962 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1e0f5a979494fb2be312a2f0470141a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1e0f5a979494fb2be312a2f0470141a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e0f5a979494fb2be312a2f0470141a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39781 } } }
I20260811 17:23:48.216586 9962 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1e0f5a979494fb2be312a2f0470141a [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:48.216244 9964 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1e0f5a979494fb2be312a2f0470141a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1e0f5a979494fb2be312a2f0470141a. Latest consensus state: current_term: 1 leader_uuid: "c1e0f5a979494fb2be312a2f0470141a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e0f5a979494fb2be312a2f0470141a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39781 } } }
I20260811 17:23:48.216686 9964 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1e0f5a979494fb2be312a2f0470141a [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:48.217234 9986 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:23:48.221621 9986 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:23:48.228215 9986 catalog_manager.cc:1347] Generated new cluster ID: ba65862b8e0c4f51856cc7368555c22d
I20260811 17:23:48.228328 9986 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:23:48.241612 9986 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:23:48.243139 9986 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:23:48.258680 9986 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1e0f5a979494fb2be312a2f0470141a: Generated new TSK 0
I20260811 17:23:48.259753 9986 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:23:48.274680 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 9835
2026-08-11T17:23:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1705 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:23:48 dist-test-slave-mjjr krb5kdc[9997](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:23:48 dist-test-slave-mjjr krb5kdc[9997](info): set up 2 sockets
Aug 11 17:23:48 dist-test-slave-mjjr krb5kdc[9997](info): commencing operation
krb5kdc: starting...
W20260811 17:23:49.563949 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.009s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:23:49 dist-test-slave-mjjr krb5kdc[9997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469029, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:23:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:23:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 17:23:49.707435 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34087
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:23:49.836081 10020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:49.836452 10020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:49.836570 10020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:49.840880 10020 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:23:49.841001 10020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:49.841048 10020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:49.841113 10020 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:23:49.841159 10020 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:23:49.847172 10020 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:49.848481 10020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:49.849659 10020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:49.859159 10033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:49.859321 10020 server_base.cc:1034] running on GCE node
W20260811 17:23:49.859079 10031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:49.859084 10030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:49.860229 10020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:49.861424 10020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:49.862598 10020 hybrid_clock.cc:648] HybridClock initialized: now 1786469029862572 us; error 42 us; skew 500 ppm
Aug 11 17:23:49 dist-test-slave-mjjr krb5kdc[9997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469029, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:23:49.866397 10020 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 17:23:49.867957 10020 webserver.cc:460] Webserver started at http://127.0.0.1:33207/ using document root <none> and password file <none>
I20260811 17:23:49.868794 10020 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:49.868891 10020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:49.869163 10020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:49.872323 10020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "4704e46917e14188b592b3c232a78fa9"
format_stamp: "Formatted at 2026-08-11 17:23:49 on dist-test-slave-mjjr"
server_key: "6a0b7c7f8e5ade073ccbc2cf205b7b44"
server_key_iv: "98aed6481b2d97f2db327ff1153ba1c3"
server_key_version: "encryptionkey@0"
I20260811 17:23:49.873234 10020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "4704e46917e14188b592b3c232a78fa9"
format_stamp: "Formatted at 2026-08-11 17:23:49 on dist-test-slave-mjjr"
server_key: "6a0b7c7f8e5ade073ccbc2cf205b7b44"
server_key_iv: "98aed6481b2d97f2db327ff1153ba1c3"
server_key_version: "encryptionkey@0"
I20260811 17:23:49.877615 10020 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 17:23:49.880399 10045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:49.881743 10020 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:23:49.881940 10020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "4704e46917e14188b592b3c232a78fa9"
format_stamp: "Formatted at 2026-08-11 17:23:49 on dist-test-slave-mjjr"
server_key: "6a0b7c7f8e5ade073ccbc2cf205b7b44"
server_key_iv: "98aed6481b2d97f2db327ff1153ba1c3"
server_key_version: "encryptionkey@0"
I20260811 17:23:49.882079 10020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:49.887207 10020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:49.890877 10020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:49.891106 10020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:49.899591 10020 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34087
I20260811 17:23:49.899694 10116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34087 every 8 connection(s)
I20260811 17:23:49.900785 10020 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:23:49.903982 10118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:49.906416 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 10020
I20260811 17:23:49.906587 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:23:49.906879 9846 external_mini_cluster.cc:1428] Setting key 40215655a470f42d16e1e8e50a71516e
I20260811 17:23:49.910847 10118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4704e46917e14188b592b3c232a78fa9: Bootstrap starting.
I20260811 17:23:49.912890 10122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:49.908532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60889 (local address 127.0.0.1:34087)
0811 17:23:49.909052 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:49.909069 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:49.909111 (+ 42us) server_negotiation.cc:407] Connection header received
0811 17:23:49.909955 (+ 844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:49.909976 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:49.910306 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:49.910673 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:49.911584 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:49.911589 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:49.911601 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:49.911612 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:49.911731 (+ 119us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:49.911765 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:49.911925 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:49.912001 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 17:23:49.912168 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":168,"threads_started":1}
I20260811 17:23:49.913542 10118 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4704e46917e14188b592b3c232a78fa9: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:49.914332 10118 log.cc:824] T 00000000000000000000000000000000 P 4704e46917e14188b592b3c232a78fa9: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:49.916046 9846 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:23:49.916811 10118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4704e46917e14188b592b3c232a78fa9: No bootstrap required, opened a new log
I20260811 17:23:49.920051 10122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:49.918223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40841 (local address 127.0.0.1:34087)
0811 17:23:49.918366 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:49.918371 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:49.918524 (+ 153us) server_negotiation.cc:407] Connection header received
0811 17:23:49.918603 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:49.918606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:49.918646 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:49.918738 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:49.919564 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:49.919568 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:49.919570 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:49.919583 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:49.919613 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:49.919626 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:49.919760 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:49.919871 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 17:23:49.919912 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 17:23:49.921578 10118 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4704e46917e14188b592b3c232a78fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4704e46917e14188b592b3c232a78fa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34087 } }
I20260811 17:23:49.921823 10118 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4704e46917e14188b592b3c232a78fa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:49.921887 10118 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4704e46917e14188b592b3c232a78fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4704e46917e14188b592b3c232a78fa9, State: Initialized, Role: FOLLOWER
I20260811 17:23:49.922394 10118 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4704e46917e14188b592b3c232a78fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4704e46917e14188b592b3c232a78fa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34087 } }
I20260811 17:23:49.922559 10118 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4704e46917e14188b592b3c232a78fa9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:23:49.922631 10118 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4704e46917e14188b592b3c232a78fa9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:23:49.922751 10118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4704e46917e14188b592b3c232a78fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:49.923835 10118 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4704e46917e14188b592b3c232a78fa9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4704e46917e14188b592b3c232a78fa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34087 } }
I20260811 17:23:49.924202 10118 leader_election.cc:302] T 00000000000000000000000000000000 P 4704e46917e14188b592b3c232a78fa9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4704e46917e14188b592b3c232a78fa9; no voters:
I20260811 17:23:49.924559 10118 leader_election.cc:288] T 00000000000000000000000000000000 P 4704e46917e14188b592b3c232a78fa9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:23:49.924728 10132 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4704e46917e14188b592b3c232a78fa9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:23:49.924985 10132 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4704e46917e14188b592b3c232a78fa9 [term 1 LEADER]: Becoming Leader. State: Replica: 4704e46917e14188b592b3c232a78fa9, State: Running, Role: LEADER
I20260811 17:23:49.925339 10132 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4704e46917e14188b592b3c232a78fa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4704e46917e14188b592b3c232a78fa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34087 } }
I20260811 17:23:49.926051 10118 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4704e46917e14188b592b3c232a78fa9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:23:49.927373 10135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4704e46917e14188b592b3c232a78fa9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4704e46917e14188b592b3c232a78fa9. Latest consensus state: current_term: 1 leader_uuid: "4704e46917e14188b592b3c232a78fa9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4704e46917e14188b592b3c232a78fa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34087 } } }
I20260811 17:23:49.927513 10135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4704e46917e14188b592b3c232a78fa9 [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:49.927835 10133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4704e46917e14188b592b3c232a78fa9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4704e46917e14188b592b3c232a78fa9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4704e46917e14188b592b3c232a78fa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34087 } } }
I20260811 17:23:49.927919 10133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4704e46917e14188b592b3c232a78fa9 [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:49.927963 10147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:23:49.931537 10147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:23:49.939281 10147 catalog_manager.cc:1347] Generated new cluster ID: 7ce7bc418e2b4772b11bb7098bdc9887
I20260811 17:23:49.939397 10147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:23:49.955183 10147 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:23:49.955631 10147 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:23:49.966069 10147 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4704e46917e14188b592b3c232a78fa9: Generated new TSK 0
I20260811 17:23:49.967103 10147 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:23:49.981516 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 10020
2026-08-11T17:23:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1701 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:23:50 dist-test-slave-mjjr krb5kdc[10168](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:23:50 dist-test-slave-mjjr krb5kdc[10168](info): set up 2 sockets
Aug 11 17:23:50 dist-test-slave-mjjr krb5kdc[10168](info): commencing operation
krb5kdc: starting...
W20260811 17:23:51.245875 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:23:51 dist-test-slave-mjjr krb5kdc[10168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:23:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:23:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 17:23:51.380569 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35365
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:23:51.498238 10188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:51.498553 10188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:51.498598 10188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:51.502839 10188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:23:51.502926 10188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:51.502946 10188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:51.502974 10188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:23:51.502990 10188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:23:51.508667 10188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:51.509912 10188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:51.511051 10188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:23:51.520540 10188 server_base.cc:1034] running on GCE node
W20260811 17:23:51.520481 10202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:51.520502 10205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:51.520481 10197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:51.521564 10188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:51.522897 10188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:51.524118 10188 hybrid_clock.cc:648] HybridClock initialized: now 1786469031524093 us; error 42 us; skew 500 ppm
Aug 11 17:23:51 dist-test-slave-mjjr krb5kdc[10168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469031, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:23:51.531775 10188 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 17:23:51.532873 10188 webserver.cc:460] Webserver started at http://127.0.0.1:42713/ using document root <none> and password file <none>
I20260811 17:23:51.533526 10188 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:51.533632 10188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:51.533896 10188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:51.536259 10188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "f7c02f89fb784390aa5af834f6da588e"
format_stamp: "Formatted at 2026-08-11 17:23:51 on dist-test-slave-mjjr"
server_key: "d812ac63eca10bc8d1b85d8321d2b6d4"
server_key_iv: "95701e661858614006a6e0f248fd7a75"
server_key_version: "encryptionkey@0"
I20260811 17:23:51.536892 10188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "f7c02f89fb784390aa5af834f6da588e"
format_stamp: "Formatted at 2026-08-11 17:23:51 on dist-test-slave-mjjr"
server_key: "d812ac63eca10bc8d1b85d8321d2b6d4"
server_key_iv: "95701e661858614006a6e0f248fd7a75"
server_key_version: "encryptionkey@0"
I20260811 17:23:51.541199 10188 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:23:51.544063 10215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:51.545562 10188 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 17:23:51.545759 10188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "f7c02f89fb784390aa5af834f6da588e"
format_stamp: "Formatted at 2026-08-11 17:23:51 on dist-test-slave-mjjr"
server_key: "d812ac63eca10bc8d1b85d8321d2b6d4"
server_key_iv: "95701e661858614006a6e0f248fd7a75"
server_key_version: "encryptionkey@0"
I20260811 17:23:51.545908 10188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:51.573458 10188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:51.577194 10188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:51.577440 10188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:51.585480 10188 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35365
I20260811 17:23:51.585510 10294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35365 every 8 connection(s)
I20260811 17:23:51.586611 10188 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:23:51.588945 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 10188
I20260811 17:23:51.589075 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:23:51.589416 9846 external_mini_cluster.cc:1428] Setting key f2388649c68b21e2fb9277a90bf89cfe
I20260811 17:23:51.589865 10295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:51.596756 10295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7c02f89fb784390aa5af834f6da588e: Bootstrap starting.
Aug 11 17:23:51 dist-test-slave-mjjr krb5kdc[10168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 17:23:51.599515 10295 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7c02f89fb784390aa5af834f6da588e: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:51.600353 10295 log.cc:824] T 00000000000000000000000000000000 P f7c02f89fb784390aa5af834f6da588e: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:51.602542 10295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7c02f89fb784390aa5af834f6da588e: No bootstrap required, opened a new log
I20260811 17:23:51.605237 10299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:51.591021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:54123 (local address 127.0.0.1:35365)
0811 17:23:51.591553 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:51.591569 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:51.591612 (+ 43us) server_negotiation.cc:407] Connection header received
0811 17:23:51.592399 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:51.592418 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:51.592792 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:51.593147 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:51.594023 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:51.595051 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:51.596046 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:51.596554 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:51.596736 (+ 182us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:51.599716 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:51.599723 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:51.599740 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:23:51.599779 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:51.601645 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:51.602242 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:51.602247 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:51.602252 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:51.602372 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:51.602782 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:51.602786 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:51.602787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:51.603327 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:51.603558 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:51.603890 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 17:23:51.604138 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":187,"threads_started":1}
I20260811 17:23:51.607304 10295 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7c02f89fb784390aa5af834f6da588e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c02f89fb784390aa5af834f6da588e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35365 } }
I20260811 17:23:51.607599 10295 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7c02f89fb784390aa5af834f6da588e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:51.607714 10295 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7c02f89fb784390aa5af834f6da588e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7c02f89fb784390aa5af834f6da588e, State: Initialized, Role: FOLLOWER
I20260811 17:23:51.608278 10295 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7c02f89fb784390aa5af834f6da588e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c02f89fb784390aa5af834f6da588e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35365 } }
I20260811 17:23:51.608453 10295 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7c02f89fb784390aa5af834f6da588e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:23:51.608525 10295 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7c02f89fb784390aa5af834f6da588e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:23:51.608634 10295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7c02f89fb784390aa5af834f6da588e [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:51.608821 9846 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:23:51.609724 10295 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7c02f89fb784390aa5af834f6da588e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c02f89fb784390aa5af834f6da588e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35365 } }
I20260811 17:23:51.610147 10295 leader_election.cc:302] T 00000000000000000000000000000000 P f7c02f89fb784390aa5af834f6da588e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7c02f89fb784390aa5af834f6da588e; no voters:
I20260811 17:23:51.610562 10295 leader_election.cc:288] T 00000000000000000000000000000000 P f7c02f89fb784390aa5af834f6da588e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:23:51.610689 10308 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7c02f89fb784390aa5af834f6da588e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:23:51.610967 10308 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7c02f89fb784390aa5af834f6da588e [term 1 LEADER]: Becoming Leader. State: Replica: f7c02f89fb784390aa5af834f6da588e, State: Running, Role: LEADER
I20260811 17:23:51.611395 10308 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7c02f89fb784390aa5af834f6da588e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c02f89fb784390aa5af834f6da588e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35365 } }
I20260811 17:23:51.612114 10295 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7c02f89fb784390aa5af834f6da588e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:23:51.612800 10311 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7c02f89fb784390aa5af834f6da588e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7c02f89fb784390aa5af834f6da588e. Latest consensus state: current_term: 1 leader_uuid: "f7c02f89fb784390aa5af834f6da588e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c02f89fb784390aa5af834f6da588e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35365 } } }
I20260811 17:23:51.613020 10311 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7c02f89fb784390aa5af834f6da588e [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:51.613250 10310 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7c02f89fb784390aa5af834f6da588e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7c02f89fb784390aa5af834f6da588e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7c02f89fb784390aa5af834f6da588e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35365 } } }
I20260811 17:23:51.613420 10310 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7c02f89fb784390aa5af834f6da588e [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:51.613561 10325 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:23:51.617152 10325 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:23:51.621407 10299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:51.611043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:45569 (local address 127.0.0.1:35365)
0811 17:23:51.611156 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:51.611161 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:51.611608 (+ 447us) server_negotiation.cc:407] Connection header received
0811 17:23:51.611695 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:51.611698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:51.611754 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:51.611851 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:51.612866 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:51.613698 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:51.614990 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:51.615388 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:51.615521 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:51.617373 (+ 1852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:51.617377 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:51.617380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:23:51.617408 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:51.619204 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:51.619878 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:51.619882 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:51.619884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:51.619984 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:51.620388 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:51.620391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:51.620393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:51.620630 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:51.620797 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:51.621137 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 17:23:51.621253 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 17:23:51.624421 10325 catalog_manager.cc:1347] Generated new cluster ID: b86f46dd0c9d474aa2826913bcf6201f
I20260811 17:23:51.624507 10325 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:23:51.639765 10325 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:23:51.641206 10325 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:23:51.648090 10325 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7c02f89fb784390aa5af834f6da588e: Generated new TSK 0
I20260811 17:23:51.648907 10325 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:23:51.661168 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 10188
W20260811 17:23:51.672526 10165 connection.cc:535] client connection to 127.0.0.1:35365 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35365): Connection reset by peer (error 104)
2026-08-11T17:23:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1680 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:23:51 dist-test-slave-mjjr krb5kdc[10342](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:23:51 dist-test-slave-mjjr krb5kdc[10342](info): set up 2 sockets
Aug 11 17:23:51 dist-test-slave-mjjr krb5kdc[10342](info): commencing operation
krb5kdc: starting...
W20260811 17:23:52.928830 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:23:53 dist-test-slave-mjjr krb5kdc[10342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469033, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:23:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:23:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 17:23:53.067411 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35157
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35157
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:23:53.191131 10366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:53.191521 10366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:53.191610 10366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:53.195924 10366 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:23:53.196055 10366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:53.196097 10366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:53.196138 10366 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:23:53.196172 10366 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:23:53.202061 10366 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35157
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:53.203399 10366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:53.204600 10366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:53.215027 10373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:53.215027 10371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:53.215073 10377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:53.215363 10366 server_base.cc:1034] running on GCE node
I20260811 17:23:53.216176 10366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:53.217502 10366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:53.218760 10366 hybrid_clock.cc:648] HybridClock initialized: now 1786469033218715 us; error 40 us; skew 500 ppm
Aug 11 17:23:53 dist-test-slave-mjjr krb5kdc[10342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469033, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:23:53.222119 10366 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 17:23:53.223232 10366 webserver.cc:460] Webserver started at http://127.0.0.1:38267/ using document root <none> and password file <none>
I20260811 17:23:53.223959 10366 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:53.224066 10366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:53.224326 10366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:53.226722 10366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "eeb6dd6004e74e419034853274d8b528"
format_stamp: "Formatted at 2026-08-11 17:23:53 on dist-test-slave-mjjr"
server_key: "0f9729e90c556ad57a4200cded3a38b9"
server_key_iv: "ecdaea7397c0a8053bb6efa8e648613e"
server_key_version: "encryptionkey@0"
I20260811 17:23:53.227381 10366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "eeb6dd6004e74e419034853274d8b528"
format_stamp: "Formatted at 2026-08-11 17:23:53 on dist-test-slave-mjjr"
server_key: "0f9729e90c556ad57a4200cded3a38b9"
server_key_iv: "ecdaea7397c0a8053bb6efa8e648613e"
server_key_version: "encryptionkey@0"
I20260811 17:23:53.231778 10366 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 17:23:53.234840 10389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:53.236552 10366 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:23:53.236785 10366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "eeb6dd6004e74e419034853274d8b528"
format_stamp: "Formatted at 2026-08-11 17:23:53 on dist-test-slave-mjjr"
server_key: "0f9729e90c556ad57a4200cded3a38b9"
server_key_iv: "ecdaea7397c0a8053bb6efa8e648613e"
server_key_version: "encryptionkey@0"
I20260811 17:23:53.236967 10366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:53.268607 10366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:53.272382 10366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:53.272665 10366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:53.281031 10366 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35157
I20260811 17:23:53.281057 10476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35157 every 8 connection(s)
I20260811 17:23:53.282089 10366 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:23:53.285429 10477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:53.286708 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 10366
I20260811 17:23:53.286836 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:23:53.287191 9846 external_mini_cluster.cc:1428] Setting key 25bd03c3267f40ff50682ae7c7101293
I20260811 17:23:53.293766 10477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eeb6dd6004e74e419034853274d8b528: Bootstrap starting.
I20260811 17:23:53.297024 10477 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eeb6dd6004e74e419034853274d8b528: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:53.298270 10477 log.cc:824] T 00000000000000000000000000000000 P eeb6dd6004e74e419034853274d8b528: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:53.299409 10481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:53.288896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:37955 (local address 127.0.0.1:35157)
0811 17:23:53.289431 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:53.289455 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:53.289508 (+ 53us) server_negotiation.cc:407] Connection header received
0811 17:23:53.290466 (+ 958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:53.290495 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:53.291009 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:53.291443 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:53.291989 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:53.293697 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:53.294852 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:53.295700 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:53.295894 (+ 194us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:53.296959 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:53.296967 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:53.296991 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:53.297712 (+ 721us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:53.297790 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:53.297823 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:53.297995 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:53.298236 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 17:23:53.298399 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":148,"threads_started":1}
I20260811 17:23:53.300936 10477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eeb6dd6004e74e419034853274d8b528: No bootstrap required, opened a new log
I20260811 17:23:53.302901 9846 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:23:53.305297 10477 raft_consensus.cc:348] T 00000000000000000000000000000000 P eeb6dd6004e74e419034853274d8b528 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeb6dd6004e74e419034853274d8b528" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35157 } }
I20260811 17:23:53.305593 10477 raft_consensus.cc:374] T 00000000000000000000000000000000 P eeb6dd6004e74e419034853274d8b528 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:53.305663 10477 raft_consensus.cc:729] T 00000000000000000000000000000000 P eeb6dd6004e74e419034853274d8b528 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eeb6dd6004e74e419034853274d8b528, State: Initialized, Role: FOLLOWER
I20260811 17:23:53.306218 10477 consensus_queue.cc:260] T 00000000000000000000000000000000 P eeb6dd6004e74e419034853274d8b528 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeb6dd6004e74e419034853274d8b528" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35157 } }
I20260811 17:23:53.306429 10477 raft_consensus.cc:388] T 00000000000000000000000000000000 P eeb6dd6004e74e419034853274d8b528 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:23:53.306529 10477 raft_consensus.cc:482] T 00000000000000000000000000000000 P eeb6dd6004e74e419034853274d8b528 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:23:53.306643 10477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eeb6dd6004e74e419034853274d8b528 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:53.307920 10477 raft_consensus.cc:504] T 00000000000000000000000000000000 P eeb6dd6004e74e419034853274d8b528 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeb6dd6004e74e419034853274d8b528" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35157 } }
I20260811 17:23:53.308364 10477 leader_election.cc:302] T 00000000000000000000000000000000 P eeb6dd6004e74e419034853274d8b528 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eeb6dd6004e74e419034853274d8b528; no voters:
I20260811 17:23:53.308784 10477 leader_election.cc:288] T 00000000000000000000000000000000 P eeb6dd6004e74e419034853274d8b528 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:23:53.308990 10492 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eeb6dd6004e74e419034853274d8b528 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:23:53.309265 10492 raft_consensus.cc:686] T 00000000000000000000000000000000 P eeb6dd6004e74e419034853274d8b528 [term 1 LEADER]: Becoming Leader. State: Replica: eeb6dd6004e74e419034853274d8b528, State: Running, Role: LEADER
I20260811 17:23:53.309707 10492 consensus_queue.cc:237] T 00000000000000000000000000000000 P eeb6dd6004e74e419034853274d8b528 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeb6dd6004e74e419034853274d8b528" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35157 } }
I20260811 17:23:53.310465 10477 sys_catalog.cc:564] T 00000000000000000000000000000000 P eeb6dd6004e74e419034853274d8b528 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:23:53.310587 10481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:53.305202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:57127 (local address 127.0.0.1:35157)
0811 17:23:53.305351 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:53.305358 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:53.305577 (+ 219us) server_negotiation.cc:407] Connection header received
0811 17:23:53.305791 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:53.305794 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:53.305854 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:53.305960 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:53.306321 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:53.307101 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:53.308337 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:53.308708 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:53.308837 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:53.310005 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:53.310009 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:53.310013 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:53.310030 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:53.310063 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:53.310077 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:53.310175 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:53.310353 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 17:23:53.310395 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 17:23:53.311127 10492 sys_catalog.cc:455] T 00000000000000000000000000000000 P eeb6dd6004e74e419034853274d8b528 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eeb6dd6004e74e419034853274d8b528. Latest consensus state: current_term: 1 leader_uuid: "eeb6dd6004e74e419034853274d8b528" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeb6dd6004e74e419034853274d8b528" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35157 } } }
I20260811 17:23:53.311246 10492 sys_catalog.cc:458] T 00000000000000000000000000000000 P eeb6dd6004e74e419034853274d8b528 [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:53.311630 10494 sys_catalog.cc:455] T 00000000000000000000000000000000 P eeb6dd6004e74e419034853274d8b528 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eeb6dd6004e74e419034853274d8b528" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeb6dd6004e74e419034853274d8b528" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35157 } } }
I20260811 17:23:53.311764 10494 sys_catalog.cc:458] T 00000000000000000000000000000000 P eeb6dd6004e74e419034853274d8b528 [sys.catalog]: This master's current role is: LEADER
W20260811 17:23:53.315748 10516 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eeb6dd6004e74e419034853274d8b528: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:23:53.315826 10516 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:23:53.315953 10500 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:23:53.316820 10500 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:23:53.322952 10500 catalog_manager.cc:1347] Generated new cluster ID: 0eeeac02fbca47818da63e96201f8520
I20260811 17:23:53.323045 10500 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:23:53.338778 10500 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:23:53.340179 10500 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:23:53.348230 10500 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eeb6dd6004e74e419034853274d8b528: Generated new TSK 0
I20260811 17:23:53.349141 10500 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:23:53.354817 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 10366
W20260811 17:23:53.372588 10340 connection.cc:535] client connection to 127.0.0.1:35157 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35157): Connection reset by peer (error 104)
2026-08-11T17:23:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1700 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:23:53 dist-test-slave-mjjr krb5kdc[10523](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:23:53 dist-test-slave-mjjr krb5kdc[10523](info): set up 2 sockets
Aug 11 17:23:53 dist-test-slave-mjjr krb5kdc[10523](info): commencing operation
krb5kdc: starting...
W20260811 17:23:54.629607 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:23:54 dist-test-slave-mjjr krb5kdc[10523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:23:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:23:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 17:23:54.774878 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34847
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34847
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:23:54.906668 10545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:54.906996 10545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:54.907040 10545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:54.911309 10545 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:23:54.911415 10545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:54.911437 10545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:54.911453 10545 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:23:54.911469 10545 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:23:54.917255 10545 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:54.918805 10545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:54.920138 10545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:54.929561 10558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:54.929612 10560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:54.929567 10563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:54.929977 10545 server_base.cc:1034] running on GCE node
I20260811 17:23:54.930666 10545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:54.932055 10545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:54.933264 10545 hybrid_clock.cc:648] HybridClock initialized: now 1786469034933235 us; error 41 us; skew 500 ppm
Aug 11 17:23:54 dist-test-slave-mjjr krb5kdc[10523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469034, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:23:54.936452 10545 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 17:23:54.937461 10545 webserver.cc:460] Webserver started at http://127.0.0.1:35939/ using document root <none> and password file <none>
I20260811 17:23:54.938056 10545 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:54.938122 10545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:54.938313 10545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:54.940667 10545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "b89220b79cd844dab4398a11872fe53a"
format_stamp: "Formatted at 2026-08-11 17:23:54 on dist-test-slave-mjjr"
server_key: "d2999fe585774a87a98f87028c25be98"
server_key_iv: "a9b6a36ccc0bc2cb20e2e09063542ce9"
server_key_version: "encryptionkey@0"
I20260811 17:23:54.941270 10545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "b89220b79cd844dab4398a11872fe53a"
format_stamp: "Formatted at 2026-08-11 17:23:54 on dist-test-slave-mjjr"
server_key: "d2999fe585774a87a98f87028c25be98"
server_key_iv: "a9b6a36ccc0bc2cb20e2e09063542ce9"
server_key_version: "encryptionkey@0"
I20260811 17:23:54.945601 10545 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:23:54.948575 10573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:54.949970 10545 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 17:23:54.950104 10545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "b89220b79cd844dab4398a11872fe53a"
format_stamp: "Formatted at 2026-08-11 17:23:54 on dist-test-slave-mjjr"
server_key: "d2999fe585774a87a98f87028c25be98"
server_key_iv: "a9b6a36ccc0bc2cb20e2e09063542ce9"
server_key_version: "encryptionkey@0"
I20260811 17:23:54.950217 10545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:54.955330 10545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:54.959061 10545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:54.959249 10545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:54.967200 10545 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34847
I20260811 17:23:54.967267 10672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34847 every 8 connection(s)
I20260811 17:23:54.968264 10545 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:23:54.971342 10676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:54.974109 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 10545
I20260811 17:23:54.974277 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:23:54.974591 9846 external_mini_cluster.cc:1428] Setting key f8b3b5cfaf5d60ad83a5ad28a60f94b2
I20260811 17:23:54.977983 10676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b89220b79cd844dab4398a11872fe53a: Bootstrap starting.
I20260811 17:23:54.980372 10682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:54.976182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:46835 (local address 127.0.0.1:34847)
0811 17:23:54.976675 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:54.976692 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:54.976729 (+ 37us) server_negotiation.cc:407] Connection header received
0811 17:23:54.977534 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:54.977555 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:54.977862 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:54.978215 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:54.979041 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:54.979046 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:54.979058 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:54.979069 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:54.979124 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:54.979144 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:54.979307 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:54.979388 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 17:23:54.979558 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":159,"threads_started":1}
I20260811 17:23:54.980795 10676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b89220b79cd844dab4398a11872fe53a: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:54.981600 10676 log.cc:824] T 00000000000000000000000000000000 P b89220b79cd844dab4398a11872fe53a: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:54.983364 9846 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:23:54.983935 10676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b89220b79cd844dab4398a11872fe53a: No bootstrap required, opened a new log
I20260811 17:23:54.987447 10682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:54.985622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:46481 (local address 127.0.0.1:34847)
0811 17:23:54.985767 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:54.985774 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:54.985973 (+ 199us) server_negotiation.cc:407] Connection header received
0811 17:23:54.986102 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:54.986104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:54.986135 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:54.986201 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:54.987002 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:54.987006 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:54.987008 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:54.987019 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:54.987046 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:54.987056 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:54.987138 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:54.987258 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 17:23:54.987296 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 17:23:54.988735 10676 raft_consensus.cc:348] T 00000000000000000000000000000000 P b89220b79cd844dab4398a11872fe53a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b89220b79cd844dab4398a11872fe53a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34847 } }
I20260811 17:23:54.989001 10676 raft_consensus.cc:374] T 00000000000000000000000000000000 P b89220b79cd844dab4398a11872fe53a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:54.989066 10676 raft_consensus.cc:729] T 00000000000000000000000000000000 P b89220b79cd844dab4398a11872fe53a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b89220b79cd844dab4398a11872fe53a, State: Initialized, Role: FOLLOWER
I20260811 17:23:54.989542 10676 consensus_queue.cc:260] T 00000000000000000000000000000000 P b89220b79cd844dab4398a11872fe53a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b89220b79cd844dab4398a11872fe53a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34847 } }
I20260811 17:23:54.989677 10676 raft_consensus.cc:388] T 00000000000000000000000000000000 P b89220b79cd844dab4398a11872fe53a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:23:54.989773 10676 raft_consensus.cc:482] T 00000000000000000000000000000000 P b89220b79cd844dab4398a11872fe53a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:23:54.989903 10676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b89220b79cd844dab4398a11872fe53a [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:54.990981 10676 raft_consensus.cc:504] T 00000000000000000000000000000000 P b89220b79cd844dab4398a11872fe53a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b89220b79cd844dab4398a11872fe53a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34847 } }
I20260811 17:23:54.991392 10676 leader_election.cc:302] T 00000000000000000000000000000000 P b89220b79cd844dab4398a11872fe53a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b89220b79cd844dab4398a11872fe53a; no voters:
I20260811 17:23:54.991825 10676 leader_election.cc:288] T 00000000000000000000000000000000 P b89220b79cd844dab4398a11872fe53a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:23:54.991961 10693 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b89220b79cd844dab4398a11872fe53a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:23:54.992234 10693 raft_consensus.cc:686] T 00000000000000000000000000000000 P b89220b79cd844dab4398a11872fe53a [term 1 LEADER]: Becoming Leader. State: Replica: b89220b79cd844dab4398a11872fe53a, State: Running, Role: LEADER
I20260811 17:23:54.992630 10693 consensus_queue.cc:237] T 00000000000000000000000000000000 P b89220b79cd844dab4398a11872fe53a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b89220b79cd844dab4398a11872fe53a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34847 } }
I20260811 17:23:54.993254 10676 sys_catalog.cc:564] T 00000000000000000000000000000000 P b89220b79cd844dab4398a11872fe53a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:23:54.994576 10695 sys_catalog.cc:455] T 00000000000000000000000000000000 P b89220b79cd844dab4398a11872fe53a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b89220b79cd844dab4398a11872fe53a. Latest consensus state: current_term: 1 leader_uuid: "b89220b79cd844dab4398a11872fe53a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b89220b79cd844dab4398a11872fe53a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34847 } } }
I20260811 17:23:54.994731 10695 sys_catalog.cc:458] T 00000000000000000000000000000000 P b89220b79cd844dab4398a11872fe53a [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:54.995224 10706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:23:54.995213 10694 sys_catalog.cc:455] T 00000000000000000000000000000000 P b89220b79cd844dab4398a11872fe53a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b89220b79cd844dab4398a11872fe53a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b89220b79cd844dab4398a11872fe53a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34847 } } }
I20260811 17:23:54.995386 10694 sys_catalog.cc:458] T 00000000000000000000000000000000 P b89220b79cd844dab4398a11872fe53a [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:54.998852 10706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:23:55.004470 10706 catalog_manager.cc:1347] Generated new cluster ID: 45ddc4f01d064ea99fa5acc62f063179
I20260811 17:23:55.004556 10706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:23:55.023669 10706 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:23:55.024073 10706 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:23:55.032308 10706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b89220b79cd844dab4398a11872fe53a: Generated new TSK 0
I20260811 17:23:55.033264 10706 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:23:55.046371 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 10545
2026-08-11T17:23:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1693 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:23:55 dist-test-slave-mjjr krb5kdc[10729](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:23:55 dist-test-slave-mjjr krb5kdc[10729](info): set up 2 sockets
Aug 11 17:23:55 dist-test-slave-mjjr krb5kdc[10729](info): commencing operation
krb5kdc: starting...
W20260811 17:23:56.313381 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:23:56 dist-test-slave-mjjr krb5kdc[10729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:23:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:23:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 17:23:56.460922 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35887
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35887
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:23:56.589020 10752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:56.589315 10752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:56.589361 10752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:56.593611 10752 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:23:56.593689 10752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:56.593710 10752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:56.593727 10752 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:23:56.593743 10752 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:23:56.599510 10752 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35887
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:56.600813 10752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:56.601950 10752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:56.611310 10762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:56.611325 10760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:56.611490 10752 server_base.cc:1034] running on GCE node
W20260811 17:23:56.611325 10759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:56.612205 10752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:56.613464 10752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:56.614648 10752 hybrid_clock.cc:648] HybridClock initialized: now 1786469036614623 us; error 38 us; skew 500 ppm
Aug 11 17:23:56 dist-test-slave-mjjr krb5kdc[10729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469036, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:23:56.621155 10752 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 17:23:56.622231 10752 webserver.cc:460] Webserver started at http://127.0.0.1:40353/ using document root <none> and password file <none>
I20260811 17:23:56.622917 10752 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:56.622990 10752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:56.623239 10752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:56.625598 10752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "357b8956b7af4d3a889ddd059a2afb6c"
format_stamp: "Formatted at 2026-08-11 17:23:56 on dist-test-slave-mjjr"
server_key: "3a88aeaf7ea0f84d24eb476e09c729f8"
server_key_iv: "3bfbcc8e64003928d6161f3cb1a31d9e"
server_key_version: "encryptionkey@0"
I20260811 17:23:56.626274 10752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "357b8956b7af4d3a889ddd059a2afb6c"
format_stamp: "Formatted at 2026-08-11 17:23:56 on dist-test-slave-mjjr"
server_key: "3a88aeaf7ea0f84d24eb476e09c729f8"
server_key_iv: "3bfbcc8e64003928d6161f3cb1a31d9e"
server_key_version: "encryptionkey@0"
I20260811 17:23:56.630620 10752 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 17:23:56.633351 10777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:56.634630 10752 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 17:23:56.634790 10752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "357b8956b7af4d3a889ddd059a2afb6c"
format_stamp: "Formatted at 2026-08-11 17:23:56 on dist-test-slave-mjjr"
server_key: "3a88aeaf7ea0f84d24eb476e09c729f8"
server_key_iv: "3bfbcc8e64003928d6161f3cb1a31d9e"
server_key_version: "encryptionkey@0"
I20260811 17:23:56.634917 10752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:56.656646 10752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:56.660319 10752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:56.660544 10752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:56.668340 10752 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35887
I20260811 17:23:56.668368 10856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35887 every 8 connection(s)
I20260811 17:23:56.669400 10752 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:23:56.670373 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 10752
I20260811 17:23:56.670513 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:23:56.670822 9846 external_mini_cluster.cc:1428] Setting key 10a28485548ad2670ec16d4423ed03d2
I20260811 17:23:56.672783 10858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:56.680163 10858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 357b8956b7af4d3a889ddd059a2afb6c: Bootstrap starting.
Aug 11 17:23:56 dist-test-slave-mjjr krb5kdc[10729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 17:23:56.683235 10858 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 357b8956b7af4d3a889ddd059a2afb6c: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:56.684262 10858 log.cc:824] T 00000000000000000000000000000000 P 357b8956b7af4d3a889ddd059a2afb6c: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:56.686867 10858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 357b8956b7af4d3a889ddd059a2afb6c: No bootstrap required, opened a new log
I20260811 17:23:56.687665 10863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:56.672364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:41229 (local address 127.0.0.1:35887)
0811 17:23:56.672845 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:56.672866 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:56.672912 (+ 46us) server_negotiation.cc:407] Connection header received
0811 17:23:56.673783 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:56.673808 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:56.674284 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:56.674671 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:56.675432 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:56.677032 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:56.677997 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:56.678697 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:56.678886 (+ 189us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:56.682002 (+ 3116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:56.682008 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:56.682026 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:23:56.682063 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:56.684176 (+ 2113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:56.684847 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:56.684852 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:56.684857 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:56.684970 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:56.685365 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:56.685369 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:56.685371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:56.685922 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:56.686142 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:56.686414 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 17:23:56.686671 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":130,"threads_started":1}
I20260811 17:23:56.690917 9846 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:23:56.691411 10858 raft_consensus.cc:348] T 00000000000000000000000000000000 P 357b8956b7af4d3a889ddd059a2afb6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "357b8956b7af4d3a889ddd059a2afb6c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35887 } }
I20260811 17:23:56.691689 10858 raft_consensus.cc:374] T 00000000000000000000000000000000 P 357b8956b7af4d3a889ddd059a2afb6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:56.691776 10858 raft_consensus.cc:729] T 00000000000000000000000000000000 P 357b8956b7af4d3a889ddd059a2afb6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 357b8956b7af4d3a889ddd059a2afb6c, State: Initialized, Role: FOLLOWER
I20260811 17:23:56.692298 10858 consensus_queue.cc:260] T 00000000000000000000000000000000 P 357b8956b7af4d3a889ddd059a2afb6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "357b8956b7af4d3a889ddd059a2afb6c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35887 } }
I20260811 17:23:56.692477 10858 raft_consensus.cc:388] T 00000000000000000000000000000000 P 357b8956b7af4d3a889ddd059a2afb6c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:23:56.692571 10858 raft_consensus.cc:482] T 00000000000000000000000000000000 P 357b8956b7af4d3a889ddd059a2afb6c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:23:56.692677 10858 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 357b8956b7af4d3a889ddd059a2afb6c [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:56.694010 10858 raft_consensus.cc:504] T 00000000000000000000000000000000 P 357b8956b7af4d3a889ddd059a2afb6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "357b8956b7af4d3a889ddd059a2afb6c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35887 } }
I20260811 17:23:56.694446 10858 leader_election.cc:302] T 00000000000000000000000000000000 P 357b8956b7af4d3a889ddd059a2afb6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 357b8956b7af4d3a889ddd059a2afb6c; no voters:
I20260811 17:23:56.694823 10858 leader_election.cc:288] T 00000000000000000000000000000000 P 357b8956b7af4d3a889ddd059a2afb6c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:23:56.694895 10876 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 357b8956b7af4d3a889ddd059a2afb6c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:23:56.695123 10876 raft_consensus.cc:686] T 00000000000000000000000000000000 P 357b8956b7af4d3a889ddd059a2afb6c [term 1 LEADER]: Becoming Leader. State: Replica: 357b8956b7af4d3a889ddd059a2afb6c, State: Running, Role: LEADER
I20260811 17:23:56.695492 10876 consensus_queue.cc:237] T 00000000000000000000000000000000 P 357b8956b7af4d3a889ddd059a2afb6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "357b8956b7af4d3a889ddd059a2afb6c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35887 } }
I20260811 17:23:56.696293 10858 sys_catalog.cc:564] T 00000000000000000000000000000000 P 357b8956b7af4d3a889ddd059a2afb6c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:23:56.697417 10879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 357b8956b7af4d3a889ddd059a2afb6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 357b8956b7af4d3a889ddd059a2afb6c. Latest consensus state: current_term: 1 leader_uuid: "357b8956b7af4d3a889ddd059a2afb6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "357b8956b7af4d3a889ddd059a2afb6c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35887 } } }
I20260811 17:23:56.697829 10879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 357b8956b7af4d3a889ddd059a2afb6c [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:56.697706 10878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 357b8956b7af4d3a889ddd059a2afb6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "357b8956b7af4d3a889ddd059a2afb6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "357b8956b7af4d3a889ddd059a2afb6c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35887 } } }
I20260811 17:23:56.698175 10878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 357b8956b7af4d3a889ddd059a2afb6c [sys.catalog]: This master's current role is: LEADER
W20260811 17:23:56.702031 10900 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 357b8956b7af4d3a889ddd059a2afb6c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:23:56.702165 10900 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:23:56.702261 10894 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:23:56.702924 10863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:56.693379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:51661 (local address 127.0.0.1:35887)
0811 17:23:56.693507 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:56.693511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:56.693630 (+ 119us) server_negotiation.cc:407] Connection header received
0811 17:23:56.693775 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:56.693778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:56.693823 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:56.693892 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:56.694755 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:56.695497 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:56.696552 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:56.696857 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:56.696960 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:56.699128 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:56.699131 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:56.699134 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:23:56.699157 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:56.700806 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:56.701321 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:56.701325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:56.701327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:56.701415 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:23:56.701767 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:23:56.701770 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:23:56.701772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:23:56.701960 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:56.702120 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:56.702721 (+ 601us) server_negotiation.cc:299] Negotiation successful
0811 17:23:56.702813 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 17:23:56.703231 10894 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:23:56.710274 10894 catalog_manager.cc:1347] Generated new cluster ID: 760c4af4c6f64e52810f5cccf1ce22ee
I20260811 17:23:56.710394 10894 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:23:56.724802 10894 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:23:56.726253 10894 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:23:56.738888 10894 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 357b8956b7af4d3a889ddd059a2afb6c: Generated new TSK 0
I20260811 17:23:56.739928 10894 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:23:56.746677 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 10752
W20260811 17:23:56.764392 10727 connection.cc:535] client connection to 127.0.0.1:35887 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35887): Connection reset by peer (error 104)
2026-08-11T17:23:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1698 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:23:56 dist-test-slave-mjjr krb5kdc[10913](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:23:56 dist-test-slave-mjjr krb5kdc[10913](info): set up 2 sockets
Aug 11 17:23:56 dist-test-slave-mjjr krb5kdc[10913](info): commencing operation
krb5kdc: starting...
W20260811 17:23:58.036638 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.257s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:23:58 dist-test-slave-mjjr krb5kdc[10913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:23:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:23:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 17:23:58.183259 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45195
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:23:58.310527 10931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:23:58.310931 10931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:23:58.311033 10931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:23:58.315634 10931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:23:58.315786 10931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:23:58.315822 10931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:23:58.315866 10931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:23:58.315901 10931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:23:58.321907 10931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45195
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:23:58.323160 10931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:23:58.324293 10931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:23:58.333505 10940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:58.333686 10931 server_base.cc:1034] running on GCE node
W20260811 17:23:58.333494 10942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:23:58.333562 10939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:23:58.334515 10931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:23:58.335884 10931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:23:58.337088 10931 hybrid_clock.cc:648] HybridClock initialized: now 1786469038337067 us; error 37 us; skew 500 ppm
Aug 11 17:23:58 dist-test-slave-mjjr krb5kdc[10913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469038, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:23:58.340417 10931 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 17:23:58.341636 10931 webserver.cc:460] Webserver started at http://127.0.0.1:33483/ using document root <none> and password file <none>
I20260811 17:23:58.342226 10931 fs_manager.cc:359] Metadata directory not provided
I20260811 17:23:58.342290 10931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:23:58.342483 10931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:23:58.344880 10931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "04e7a615e1334958a5d703a6ce1f9f8b"
format_stamp: "Formatted at 2026-08-11 17:23:58 on dist-test-slave-mjjr"
server_key: "e572b606f58e1c18add817d19d7b9115"
server_key_iv: "43eb1c0942e940e5c4e1815a7b18af45"
server_key_version: "encryptionkey@0"
I20260811 17:23:58.345458 10931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "04e7a615e1334958a5d703a6ce1f9f8b"
format_stamp: "Formatted at 2026-08-11 17:23:58 on dist-test-slave-mjjr"
server_key: "e572b606f58e1c18add817d19d7b9115"
server_key_iv: "43eb1c0942e940e5c4e1815a7b18af45"
server_key_version: "encryptionkey@0"
I20260811 17:23:58.349776 10931 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:23:58.352480 10950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:23:58.353866 10931 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:23:58.353988 10931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "04e7a615e1334958a5d703a6ce1f9f8b"
format_stamp: "Formatted at 2026-08-11 17:23:58 on dist-test-slave-mjjr"
server_key: "e572b606f58e1c18add817d19d7b9115"
server_key_iv: "43eb1c0942e940e5c4e1815a7b18af45"
server_key_version: "encryptionkey@0"
I20260811 17:23:58.354094 10931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:23:58.366799 10931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:23:58.370373 10931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:23:58.370561 10931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:23:58.380875 10931 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45195
I20260811 17:23:58.380929 11033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45195 every 8 connection(s)
I20260811 17:23:58.382117 10931 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:23:58.383565 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 10931
I20260811 17:23:58.383816 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:23:58.384145 9846 external_mini_cluster.cc:1428] Setting key cf589c2cdfa4363287f23dfbb751bb3f
I20260811 17:23:58.386008 11034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:23:58.393246 11034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04e7a615e1334958a5d703a6ce1f9f8b: Bootstrap starting.
I20260811 17:23:58.394305 11038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:58.386054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:43625 (local address 127.0.0.1:45195)
0811 17:23:58.386620 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:58.386637 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:58.386678 (+ 41us) server_negotiation.cc:407] Connection header received
0811 17:23:58.387356 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:58.387373 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:58.387844 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:58.388153 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:58.388524 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:58.389562 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:58.390778 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:58.391260 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:58.391382 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:58.392428 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:58.392434 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:58.392447 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:58.392903 (+ 456us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:58.392963 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:58.392988 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:58.393098 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:58.393324 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 17:23:58.393515 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":171,"threads_started":1}
I20260811 17:23:58.396057 11034 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04e7a615e1334958a5d703a6ce1f9f8b: Neither blocks nor log segments found. Creating new log.
I20260811 17:23:58.397046 11034 log.cc:824] T 00000000000000000000000000000000 P 04e7a615e1334958a5d703a6ce1f9f8b: Log is configured to *not* fsync() on all Append() calls
I20260811 17:23:58.397969 9846 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:23:58.399564 11034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04e7a615e1334958a5d703a6ce1f9f8b: No bootstrap required, opened a new log
I20260811 17:23:58.403020 11034 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04e7a615e1334958a5d703a6ce1f9f8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e7a615e1334958a5d703a6ce1f9f8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45195 } }
I20260811 17:23:58.403332 11034 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04e7a615e1334958a5d703a6ce1f9f8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:23:58.403398 11034 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04e7a615e1334958a5d703a6ce1f9f8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04e7a615e1334958a5d703a6ce1f9f8b, State: Initialized, Role: FOLLOWER
I20260811 17:23:58.403921 11034 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04e7a615e1334958a5d703a6ce1f9f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e7a615e1334958a5d703a6ce1f9f8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45195 } }
I20260811 17:23:58.404089 11034 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04e7a615e1334958a5d703a6ce1f9f8b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:23:58.404165 11034 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04e7a615e1334958a5d703a6ce1f9f8b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:23:58.404296 11034 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04e7a615e1334958a5d703a6ce1f9f8b [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:23:58.404897 11038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:23:58.400290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:40229 (local address 127.0.0.1:45195)
0811 17:23:58.400437 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 17:23:58.400442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:23:58.400597 (+ 155us) server_negotiation.cc:407] Connection header received
0811 17:23:58.400803 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:23:58.400806 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:23:58.400853 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:23:58.400949 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:23:58.401239 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:58.402167 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:58.403033 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:23:58.403272 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:23:58.403388 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:23:58.404371 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:23:58.404376 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:23:58.404378 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:23:58.404411 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:23:58.404444 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:23:58.404458 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:23:58.404568 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 17:23:58.404696 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 17:23:58.404732 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 17:23:58.405659 11034 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04e7a615e1334958a5d703a6ce1f9f8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e7a615e1334958a5d703a6ce1f9f8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45195 } }
I20260811 17:23:58.406069 11034 leader_election.cc:302] T 00000000000000000000000000000000 P 04e7a615e1334958a5d703a6ce1f9f8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04e7a615e1334958a5d703a6ce1f9f8b; no voters:
I20260811 17:23:58.406497 11034 leader_election.cc:288] T 00000000000000000000000000000000 P 04e7a615e1334958a5d703a6ce1f9f8b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:23:58.406639 11053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04e7a615e1334958a5d703a6ce1f9f8b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:23:58.406899 11053 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04e7a615e1334958a5d703a6ce1f9f8b [term 1 LEADER]: Becoming Leader. State: Replica: 04e7a615e1334958a5d703a6ce1f9f8b, State: Running, Role: LEADER
I20260811 17:23:58.407222 11053 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04e7a615e1334958a5d703a6ce1f9f8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e7a615e1334958a5d703a6ce1f9f8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45195 } }
I20260811 17:23:58.407949 11034 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04e7a615e1334958a5d703a6ce1f9f8b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:23:58.410090 11055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04e7a615e1334958a5d703a6ce1f9f8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04e7a615e1334958a5d703a6ce1f9f8b. Latest consensus state: current_term: 1 leader_uuid: "04e7a615e1334958a5d703a6ce1f9f8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e7a615e1334958a5d703a6ce1f9f8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45195 } } }
I20260811 17:23:58.410229 11055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04e7a615e1334958a5d703a6ce1f9f8b [sys.catalog]: This master's current role is: LEADER
I20260811 17:23:58.410526 11054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04e7a615e1334958a5d703a6ce1f9f8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04e7a615e1334958a5d703a6ce1f9f8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e7a615e1334958a5d703a6ce1f9f8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45195 } } }
I20260811 17:23:58.410885 11054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04e7a615e1334958a5d703a6ce1f9f8b [sys.catalog]: This master's current role is: LEADER
W20260811 17:23:58.413478 11077 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 04e7a615e1334958a5d703a6ce1f9f8b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:23:58.413559 11077 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:23:58.413648 11076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:23:58.414422 11076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:23:58.420599 11076 catalog_manager.cc:1347] Generated new cluster ID: b8e2d6196118475bb9de27cfc7fc2123
I20260811 17:23:58.420691 11076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:23:58.443342 11076 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:23:58.444533 11076 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:23:58.454548 11076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04e7a615e1334958a5d703a6ce1f9f8b: Generated new TSK 0
I20260811 17:23:58.455562 11076 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:23:58.471854 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 10931
W20260811 17:23:58.488476 10911 connection.cc:535] client connection to 127.0.0.1:45195 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45195): Connection reset by peer (error 104)
2026-08-11T17:23:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1723 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:23:58 dist-test-slave-mjjr krb5kdc[11088](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:23:58 dist-test-slave-mjjr krb5kdc[11088](info): set up 2 sockets
Aug 11 17:23:58 dist-test-slave-mjjr krb5kdc[11088](info): commencing operation
krb5kdc: starting...
W20260811 17:23:59.766510 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.263s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:23:59 dist-test-slave-mjjr krb5kdc[11088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:23:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:23:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 17:23:59.920851 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35915
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:24:00.045172 11105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:24:00.045486 11105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:24:00.045529 11105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:24:00.049861 11105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:24:00.049942 11105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:24:00.049962 11105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:24:00.049980 11105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:24:00.049995 11105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:24:00.055501 11105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35915
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468743878666-9846-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:24:00.056782 11105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:24:00.057874 11105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:24:00.067219 11111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:24:00.067270 11112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:24:00.067409 11105 server_base.cc:1034] running on GCE node
W20260811 17:24:00.067260 11116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:24:00.068491 11105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:24:00.069728 11105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:24:00.070914 11105 hybrid_clock.cc:648] HybridClock initialized: now 1786469040070893 us; error 33 us; skew 500 ppm
Aug 11 17:24:00 dist-test-slave-mjjr krb5kdc[11088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469040, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:24:00.074133 11105 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 17:24:00.075201 11105 webserver.cc:460] Webserver started at http://127.0.0.1:39209/ using document root <none> and password file <none>
I20260811 17:24:00.075913 11105 fs_manager.cc:359] Metadata directory not provided
I20260811 17:24:00.075986 11105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:24:00.076226 11105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:24:00.078576 11105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "29810a9d0b4f43c6bcd9b9b32672c42d"
format_stamp: "Formatted at 2026-08-11 17:24:00 on dist-test-slave-mjjr"
server_key: "f351886b5fec3876e1143861f3180c9d"
server_key_iv: "d8c6550114b16e878ab44544d0bcc607"
server_key_version: "encryptionkey@0"
I20260811 17:24:00.079133 11105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "29810a9d0b4f43c6bcd9b9b32672c42d"
format_stamp: "Formatted at 2026-08-11 17:24:00 on dist-test-slave-mjjr"
server_key: "f351886b5fec3876e1143861f3180c9d"
server_key_iv: "d8c6550114b16e878ab44544d0bcc607"
server_key_version: "encryptionkey@0"
I20260811 17:24:00.083863 11105 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260811 17:24:00.086596 11129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:24:00.088055 11105 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:24:00.088183 11105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "29810a9d0b4f43c6bcd9b9b32672c42d"
format_stamp: "Formatted at 2026-08-11 17:24:00 on dist-test-slave-mjjr"
server_key: "f351886b5fec3876e1143861f3180c9d"
server_key_iv: "d8c6550114b16e878ab44544d0bcc607"
server_key_version: "encryptionkey@0"
I20260811 17:24:00.088290 11105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:24:00.093319 11105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:24:00.096480 11105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:24:00.096671 11105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:24:00.105357 11105 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35915
I20260811 17:24:00.105376 11214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35915 every 8 connection(s)
I20260811 17:24:00.106680 11105 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:24:00.109117 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 11105
I20260811 17:24:00.109263 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:24:00.109557 9846 external_mini_cluster.cc:1428] Setting key d97ba24175c6125ccb3e124bd93226b7
I20260811 17:24:00.110561 11215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:24:00.115832 11224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:00.111099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:55465 (local address 127.0.0.1:35915)
0811 17:24:00.111615 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:00.111634 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:00.111735 (+ 101us) server_negotiation.cc:407] Connection header received
0811 17:24:00.112559 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:00.112595 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:00.113077 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:00.113494 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:00.114386 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:00.114391 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:00.114404 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:24:00.114415 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:00.114470 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:24:00.114489 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:00.114709 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:00.114787 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 17:24:00.114955 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":153,"threads_started":1}
I20260811 17:24:00.118041 11215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29810a9d0b4f43c6bcd9b9b32672c42d: Bootstrap starting.
I20260811 17:24:00.119468 9846 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:24:00.121057 11215 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29810a9d0b4f43c6bcd9b9b32672c42d: Neither blocks nor log segments found. Creating new log.
I20260811 17:24:00.122080 11215 log.cc:824] T 00000000000000000000000000000000 P 29810a9d0b4f43c6bcd9b9b32672c42d: Log is configured to *not* fsync() on all Append() calls
I20260811 17:24:00.124362 11224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:00.122025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.116:41871 (local address 127.0.0.1:35915)
0811 17:24:00.122159 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:00.122186 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:00.122329 (+ 143us) server_negotiation.cc:407] Connection header received
0811 17:24:00.122545 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:00.122551 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:00.122611 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:00.122849 (+ 238us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:00.123788 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:00.123791 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:00.123793 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:24:00.123806 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:00.123837 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:24:00.123849 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:00.123944 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:00.124056 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 17:24:00.124096 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 17:24:00.125355 11215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29810a9d0b4f43c6bcd9b9b32672c42d: No bootstrap required, opened a new log
I20260811 17:24:00.129739 11215 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29810a9d0b4f43c6bcd9b9b32672c42d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29810a9d0b4f43c6bcd9b9b32672c42d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35915 } }
I20260811 17:24:00.130005 11215 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29810a9d0b4f43c6bcd9b9b32672c42d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:24:00.130069 11215 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29810a9d0b4f43c6bcd9b9b32672c42d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29810a9d0b4f43c6bcd9b9b32672c42d, State: Initialized, Role: FOLLOWER
I20260811 17:24:00.130546 11215 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29810a9d0b4f43c6bcd9b9b32672c42d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29810a9d0b4f43c6bcd9b9b32672c42d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35915 } }
I20260811 17:24:00.130714 11215 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29810a9d0b4f43c6bcd9b9b32672c42d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:24:00.130806 11215 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29810a9d0b4f43c6bcd9b9b32672c42d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:24:00.130936 11215 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29810a9d0b4f43c6bcd9b9b32672c42d [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:24:00.132236 11215 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29810a9d0b4f43c6bcd9b9b32672c42d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29810a9d0b4f43c6bcd9b9b32672c42d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35915 } }
I20260811 17:24:00.132643 11215 leader_election.cc:302] T 00000000000000000000000000000000 P 29810a9d0b4f43c6bcd9b9b32672c42d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29810a9d0b4f43c6bcd9b9b32672c42d; no voters:
I20260811 17:24:00.133049 11215 leader_election.cc:288] T 00000000000000000000000000000000 P 29810a9d0b4f43c6bcd9b9b32672c42d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:24:00.133328 11236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29810a9d0b4f43c6bcd9b9b32672c42d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:24:00.133594 11236 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29810a9d0b4f43c6bcd9b9b32672c42d [term 1 LEADER]: Becoming Leader. State: Replica: 29810a9d0b4f43c6bcd9b9b32672c42d, State: Running, Role: LEADER
I20260811 17:24:00.133971 11236 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29810a9d0b4f43c6bcd9b9b32672c42d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29810a9d0b4f43c6bcd9b9b32672c42d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35915 } }
I20260811 17:24:00.134508 11215 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29810a9d0b4f43c6bcd9b9b32672c42d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:24:00.135862 11240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29810a9d0b4f43c6bcd9b9b32672c42d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29810a9d0b4f43c6bcd9b9b32672c42d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29810a9d0b4f43c6bcd9b9b32672c42d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35915 } } }
I20260811 17:24:00.136014 11240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29810a9d0b4f43c6bcd9b9b32672c42d [sys.catalog]: This master's current role is: LEADER
I20260811 17:24:00.136358 11241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29810a9d0b4f43c6bcd9b9b32672c42d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29810a9d0b4f43c6bcd9b9b32672c42d. Latest consensus state: current_term: 1 leader_uuid: "29810a9d0b4f43c6bcd9b9b32672c42d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29810a9d0b4f43c6bcd9b9b32672c42d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35915 } } }
I20260811 17:24:00.136480 11241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29810a9d0b4f43c6bcd9b9b32672c42d [sys.catalog]: This master's current role is: LEADER
I20260811 17:24:00.136569 11251 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:24:00.140256 11251 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:24:00.146868 11251 catalog_manager.cc:1347] Generated new cluster ID: 4e6ff7440bf74304b5e0ab698a4b933f
I20260811 17:24:00.146997 11251 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:24:00.170356 11251 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:24:00.171025 11251 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:24:00.179656 11251 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29810a9d0b4f43c6bcd9b9b32672c42d: Generated new TSK 0
I20260811 17:24:00.180658 11251 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:24:00.189617 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 11105
2026-08-11T17:24:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1719 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:24:00 dist-test-slave-mjjr krb5kdc[11278](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:24:00 dist-test-slave-mjjr krb5kdc[11278](info): set up 2 sockets
Aug 11 17:24:00 dist-test-slave-mjjr krb5kdc[11278](info): commencing operation
krb5kdc: starting...
W20260811 17:24:01.502475 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.280s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:24:01 dist-test-slave-mjjr krb5kdc[11278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:24:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:24:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:24:01.646986 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:33673
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:35641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:33673
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:24:01.777221 11296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:24:01.777637 11296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:24:01.777731 11296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:24:01.782652 11296 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:24:01.782790 11296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:24:01.782832 11296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:24:01.782881 11296 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:24:01.782917 11296 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:24:01.789014 11296 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:35641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:33673
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:33673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:24:01.790396 11296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:24:01.791612 11296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:24:01.801569 11311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:24:01.801627 11296 server_base.cc:1034] running on GCE node
W20260811 17:24:01.801579 11308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:24:01.801569 11309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:24:01.802554 11296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:24:01.804020 11296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:24:01.805219 11296 hybrid_clock.cc:648] HybridClock initialized: now 1786469041805205 us; error 35 us; skew 500 ppm
Aug 11 17:24:01 dist-test-slave-mjjr krb5kdc[11278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469041, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:24:01.814628 11296 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:24:01.816099 11296 webserver.cc:460] Webserver started at http://127.9.157.190:43327/ using document root <none> and password file <none>
I20260811 17:24:01.816811 11296 fs_manager.cc:359] Metadata directory not provided
I20260811 17:24:01.816893 11296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:24:01.817214 11296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:24:01.819918 11296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "6ee89a03dac7414eb9fbd83755cc147b"
format_stamp: "Formatted at 2026-08-11 17:24:01 on dist-test-slave-mjjr"
server_key: "65d5d085294b0e0d7dba0b12f4f09059"
server_key_iv: "a5fdc5789a918bda9a4cda77475b3e56"
server_key_version: "encryptionkey@0"
I20260811 17:24:01.820561 11296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "6ee89a03dac7414eb9fbd83755cc147b"
format_stamp: "Formatted at 2026-08-11 17:24:01 on dist-test-slave-mjjr"
server_key: "65d5d085294b0e0d7dba0b12f4f09059"
server_key_iv: "a5fdc5789a918bda9a4cda77475b3e56"
server_key_version: "encryptionkey@0"
I20260811 17:24:01.825770 11296 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 17:24:01.828992 11323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:24:01.830790 11296 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 17:24:01.830994 11296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "6ee89a03dac7414eb9fbd83755cc147b"
format_stamp: "Formatted at 2026-08-11 17:24:01 on dist-test-slave-mjjr"
server_key: "65d5d085294b0e0d7dba0b12f4f09059"
server_key_iv: "a5fdc5789a918bda9a4cda77475b3e56"
server_key_version: "encryptionkey@0"
I20260811 17:24:01.831144 11296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:24:01.845728 11296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:24:01.849459 11296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:24:01.849704 11296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:24:01.858049 11296 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:33673
I20260811 17:24:01.858248 11423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:33673 every 8 connection(s)
I20260811 17:24:01.859236 11296 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:24:01.862501 11424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:24:01.866575 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 11296
I20260811 17:24:01.866763 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:24:01.867156 9846 external_mini_cluster.cc:1428] Setting key 4ffffaaf0361242757902138dedaba73
I20260811 17:24:01.872464 11424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ee89a03dac7414eb9fbd83755cc147b: Bootstrap starting.
I20260811 17:24:01.875047 11424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ee89a03dac7414eb9fbd83755cc147b: Neither blocks nor log segments found. Creating new log.
I20260811 17:24:01.876005 11424 log.cc:824] T 00000000000000000000000000000000 P 6ee89a03dac7414eb9fbd83755cc147b: Log is configured to *not* fsync() on all Append() calls
I20260811 17:24:01.878604 11424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ee89a03dac7414eb9fbd83755cc147b: No bootstrap required, opened a new log
Aug 11 17:24:01 dist-test-slave-mjjr krb5kdc[11278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:24:01.882575 11424 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ee89a03dac7414eb9fbd83755cc147b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee89a03dac7414eb9fbd83755cc147b" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 33673 } }
I20260811 17:24:01.882866 11424 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ee89a03dac7414eb9fbd83755cc147b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:24:01.882949 11424 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ee89a03dac7414eb9fbd83755cc147b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ee89a03dac7414eb9fbd83755cc147b, State: Initialized, Role: FOLLOWER
I20260811 17:24:01.883507 11424 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ee89a03dac7414eb9fbd83755cc147b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee89a03dac7414eb9fbd83755cc147b" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 33673 } }
I20260811 17:24:01.883689 11424 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ee89a03dac7414eb9fbd83755cc147b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:24:01.883767 11424 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ee89a03dac7414eb9fbd83755cc147b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:24:01.883894 11424 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ee89a03dac7414eb9fbd83755cc147b [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:24:01.885226 11424 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ee89a03dac7414eb9fbd83755cc147b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee89a03dac7414eb9fbd83755cc147b" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 33673 } }
I20260811 17:24:01.885639 11424 leader_election.cc:302] T 00000000000000000000000000000000 P 6ee89a03dac7414eb9fbd83755cc147b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ee89a03dac7414eb9fbd83755cc147b; no voters:
I20260811 17:24:01.886001 11433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:01.868830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47531 (local address 127.9.157.190:33673)
0811 17:24:01.869576 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:01.869601 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:01.869650 (+ 49us) server_negotiation.cc:407] Connection header received
0811 17:24:01.871195 (+ 1545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:01.871246 (+ 51us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:01.872094 (+ 848us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:01.872587 (+ 493us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:01.873852 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:01.874947 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:01.876020 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:01.876599 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:01.876789 (+ 190us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:01.879851 (+ 3062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:01.879857 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:01.879873 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:01.879908 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:01.882304 (+ 2396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:01.882952 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:01.882961 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:01.882968 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:01.883141 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:01.883511 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:01.883515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:01.883517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:01.884079 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:01.884327 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:01.884699 (+ 372us) server_negotiation.cc:299] Negotiation successful
0811 17:24:01.884979 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":168,"threads_started":1}
I20260811 17:24:01.886011 11424 leader_election.cc:288] T 00000000000000000000000000000000 P 6ee89a03dac7414eb9fbd83755cc147b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:24:01.886145 11437 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ee89a03dac7414eb9fbd83755cc147b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:24:01.886456 11437 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ee89a03dac7414eb9fbd83755cc147b [term 1 LEADER]: Becoming Leader. State: Replica: 6ee89a03dac7414eb9fbd83755cc147b, State: Running, Role: LEADER
I20260811 17:24:01.886852 11437 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ee89a03dac7414eb9fbd83755cc147b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee89a03dac7414eb9fbd83755cc147b" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 33673 } }
I20260811 17:24:01.887841 11424 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ee89a03dac7414eb9fbd83755cc147b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:24:01.889005 11438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ee89a03dac7414eb9fbd83755cc147b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ee89a03dac7414eb9fbd83755cc147b. Latest consensus state: current_term: 1 leader_uuid: "6ee89a03dac7414eb9fbd83755cc147b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee89a03dac7414eb9fbd83755cc147b" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 33673 } } }
I20260811 17:24:01.889180 11438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ee89a03dac7414eb9fbd83755cc147b [sys.catalog]: This master's current role is: LEADER
I20260811 17:24:01.890136 9846 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:24:01.889761 11443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ee89a03dac7414eb9fbd83755cc147b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ee89a03dac7414eb9fbd83755cc147b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee89a03dac7414eb9fbd83755cc147b" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 33673 } } }
I20260811 17:24:01.890475 11443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ee89a03dac7414eb9fbd83755cc147b [sys.catalog]: This master's current role is: LEADER
W20260811 17:24:01.894325 11459 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6ee89a03dac7414eb9fbd83755cc147b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:24:01.894543 11459 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:24:01.894624 11451 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:24:01.895853 11451 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:24:01.902510 11433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:01.892752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40685 (local address 127.9.157.190:33673)
0811 17:24:01.892919 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:01.892925 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:01.893298 (+ 373us) server_negotiation.cc:407] Connection header received
0811 17:24:01.893497 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:01.893500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:01.893547 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:01.893669 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:01.894414 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:01.895240 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:01.896278 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:01.896536 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:01.896648 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:01.898251 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:01.898256 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:01.898259 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:01.898288 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:01.900119 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:01.900830 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:01.900834 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:01.900836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:01.900951 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:01.901399 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:01.901403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:01.901404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:01.901646 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:01.901811 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:01.902179 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 17:24:01.902312 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 17:24:01.903086 11451 catalog_manager.cc:1347] Generated new cluster ID: efb849e068f849898bfc5fd792c602ef
I20260811 17:24:01.903162 11451 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:24:01.945101 11451 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:24:01.946367 11451 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:24:01.956938 11451 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ee89a03dac7414eb9fbd83755cc147b: Generated new TSK 0
I20260811 17:24:01.957913 11451 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:24:01.963307 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 11296
W20260811 17:24:01.981386 11275 connection.cc:535] client connection to 127.9.157.190:33673 recv error: Network error: failed to read from TLS socket (remote: 127.9.157.190:33673): Connection reset by peer (error 104)
2026-08-11T17:24:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1773 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:24:02 dist-test-slave-mjjr krb5kdc[11477](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:24:02 dist-test-slave-mjjr krb5kdc[11477](info): set up 2 sockets
Aug 11 17:24:02 dist-test-slave-mjjr krb5kdc[11477](info): commencing operation
krb5kdc: starting...
W20260811 17:24:03.274587 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:24:03 dist-test-slave-mjjr krb5kdc[11477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:24:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:24:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:24:03.422735 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:41237
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:38229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:41237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:24:03.554909 11499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:24:03.555522 11499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:24:03.555678 11499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:24:03.559967 11499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:24:03.560045 11499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:24:03.560066 11499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:24:03.560083 11499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:24:03.560130 11499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:24:03.565774 11499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:38229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:41237
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:41237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:24:03.567067 11499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:24:03.568245 11499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:24:03.577693 11504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:24:03.577693 11508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:24:03.577872 11499 server_base.cc:1034] running on GCE node
W20260811 17:24:03.577729 11515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:24:03.578630 11499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:24:03.579929 11499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:24:03.581153 11499 hybrid_clock.cc:648] HybridClock initialized: now 1786469043581132 us; error 36 us; skew 500 ppm
Aug 11 17:24:03 dist-test-slave-mjjr krb5kdc[11477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469043, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:24:03.584455 11499 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:24:03.585726 11499 webserver.cc:460] Webserver started at http://127.9.157.190:35915/ using document root <none> and password file <none>
I20260811 17:24:03.586575 11499 fs_manager.cc:359] Metadata directory not provided
I20260811 17:24:03.586716 11499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:24:03.587069 11499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:24:03.589514 11499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "7ac752b01508400ebab1a21f0b7bb2fa"
format_stamp: "Formatted at 2026-08-11 17:24:03 on dist-test-slave-mjjr"
server_key: "b3d6c36f441051275cc4357c6d535837"
server_key_iv: "68272ac8952139f5192fc82fbd91f03a"
server_key_version: "encryptionkey@0"
I20260811 17:24:03.590181 11499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "7ac752b01508400ebab1a21f0b7bb2fa"
format_stamp: "Formatted at 2026-08-11 17:24:03 on dist-test-slave-mjjr"
server_key: "b3d6c36f441051275cc4357c6d535837"
server_key_iv: "68272ac8952139f5192fc82fbd91f03a"
server_key_version: "encryptionkey@0"
I20260811 17:24:03.594920 11499 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 17:24:03.597791 11534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:24:03.599233 11499 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 17:24:03.599408 11499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "7ac752b01508400ebab1a21f0b7bb2fa"
format_stamp: "Formatted at 2026-08-11 17:24:03 on dist-test-slave-mjjr"
server_key: "b3d6c36f441051275cc4357c6d535837"
server_key_iv: "68272ac8952139f5192fc82fbd91f03a"
server_key_version: "encryptionkey@0"
I20260811 17:24:03.599546 11499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:24:03.618314 11499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:24:03.621799 11499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:24:03.622030 11499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:24:03.631067 11499 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:41237
I20260811 17:24:03.631096 11636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:41237 every 8 connection(s)
I20260811 17:24:03.632189 11499 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:24:03.635355 11637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:24:03.641675 11637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ac752b01508400ebab1a21f0b7bb2fa: Bootstrap starting.
I20260811 17:24:03.642270 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 11499
I20260811 17:24:03.642445 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:24:03.642776 9846 external_mini_cluster.cc:1428] Setting key 99fce9456e3a7b0d76ee1f564779721d
I20260811 17:24:03.644819 11637 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ac752b01508400ebab1a21f0b7bb2fa: Neither blocks nor log segments found. Creating new log.
I20260811 17:24:03.645934 11637 log.cc:824] T 00000000000000000000000000000000 P 7ac752b01508400ebab1a21f0b7bb2fa: Log is configured to *not* fsync() on all Append() calls
I20260811 17:24:03.648690 11637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ac752b01508400ebab1a21f0b7bb2fa: No bootstrap required, opened a new log
I20260811 17:24:03.653450 11637 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ac752b01508400ebab1a21f0b7bb2fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ac752b01508400ebab1a21f0b7bb2fa" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41237 } }
I20260811 17:24:03.653798 11637 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ac752b01508400ebab1a21f0b7bb2fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:24:03.653771 11645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:03.644301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39853 (local address 127.9.157.190:41237)
0811 17:24:03.644795 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:03.644817 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:03.644877 (+ 60us) server_negotiation.cc:407] Connection header received
0811 17:24:03.645856 (+ 979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:03.645886 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:03.646432 (+ 546us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:03.646908 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:03.647495 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:03.648612 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:03.649774 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:03.650446 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:03.650643 (+ 197us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:03.651750 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:03.651755 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:03.651770 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:24:03.652204 (+ 434us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:03.652267 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:24:03.652291 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:03.652448 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:03.652660 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 17:24:03.652824 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":134,"threads_started":1}
I20260811 17:24:03.653941 11637 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ac752b01508400ebab1a21f0b7bb2fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ac752b01508400ebab1a21f0b7bb2fa, State: Initialized, Role: FOLLOWER
I20260811 17:24:03.654485 11637 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ac752b01508400ebab1a21f0b7bb2fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ac752b01508400ebab1a21f0b7bb2fa" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41237 } }
I20260811 17:24:03.654656 11637 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ac752b01508400ebab1a21f0b7bb2fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:24:03.654745 11637 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ac752b01508400ebab1a21f0b7bb2fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:24:03.654897 11637 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ac752b01508400ebab1a21f0b7bb2fa [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:24:03.656428 11637 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ac752b01508400ebab1a21f0b7bb2fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ac752b01508400ebab1a21f0b7bb2fa" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41237 } }
I20260811 17:24:03.656801 11637 leader_election.cc:302] T 00000000000000000000000000000000 P 7ac752b01508400ebab1a21f0b7bb2fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ac752b01508400ebab1a21f0b7bb2fa; no voters:
I20260811 17:24:03.657182 11637 leader_election.cc:288] T 00000000000000000000000000000000 P 7ac752b01508400ebab1a21f0b7bb2fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:24:03.657298 11649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ac752b01508400ebab1a21f0b7bb2fa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:24:03.657450 9846 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:24:03.657574 11649 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ac752b01508400ebab1a21f0b7bb2fa [term 1 LEADER]: Becoming Leader. State: Replica: 7ac752b01508400ebab1a21f0b7bb2fa, State: Running, Role: LEADER
I20260811 17:24:03.657896 11649 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ac752b01508400ebab1a21f0b7bb2fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ac752b01508400ebab1a21f0b7bb2fa" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41237 } }
I20260811 17:24:03.658958 11637 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ac752b01508400ebab1a21f0b7bb2fa [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:24:03.659188 11649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ac752b01508400ebab1a21f0b7bb2fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ac752b01508400ebab1a21f0b7bb2fa. Latest consensus state: current_term: 1 leader_uuid: "7ac752b01508400ebab1a21f0b7bb2fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ac752b01508400ebab1a21f0b7bb2fa" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41237 } } }
I20260811 17:24:03.659354 11649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ac752b01508400ebab1a21f0b7bb2fa [sys.catalog]: This master's current role is: LEADER
I20260811 17:24:03.659804 11651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ac752b01508400ebab1a21f0b7bb2fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ac752b01508400ebab1a21f0b7bb2fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ac752b01508400ebab1a21f0b7bb2fa" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41237 } } }
I20260811 17:24:03.659907 11651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ac752b01508400ebab1a21f0b7bb2fa [sys.catalog]: This master's current role is: LEADER
I20260811 17:24:03.660401 11669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:24:03.664600 11669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:24:03.667850 11645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:03.661781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48013 (local address 127.9.157.190:41237)
0811 17:24:03.661921 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:03.661926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:03.662247 (+ 321us) server_negotiation.cc:407] Connection header received
0811 17:24:03.662436 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:03.662440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:03.662501 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:03.662606 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:03.662961 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:03.663738 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:03.665734 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:03.666080 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:03.666193 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:03.667277 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:03.667281 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:03.667284 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:24:03.667299 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:03.667336 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:24:03.667349 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:03.667493 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:03.667629 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 17:24:03.667688 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 17:24:03.671452 11669 catalog_manager.cc:1347] Generated new cluster ID: 3900743aacd8475f9d6031a7085a6209
I20260811 17:24:03.671541 11669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:24:03.691318 11669 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:24:03.693092 11669 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:24:03.705099 11669 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ac752b01508400ebab1a21f0b7bb2fa: Generated new TSK 0
I20260811 17:24:03.705894 11669 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:24:03.713482 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 11499
W20260811 17:24:03.729903 11475 connection.cc:535] client connection to 127.9.157.190:41237 recv error: Network error: failed to read from TLS socket (remote: 127.9.157.190:41237): Connection reset by peer (error 104)
2026-08-11T17:24:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1748 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:24:03 dist-test-slave-mjjr krb5kdc[11699](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:24:03 dist-test-slave-mjjr krb5kdc[11699](info): set up 2 sockets
Aug 11 17:24:03 dist-test-slave-mjjr krb5kdc[11699](info): commencing operation
krb5kdc: starting...
W20260811 17:24:04.999706 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:24:05 dist-test-slave-mjjr krb5kdc[11699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:24:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:24:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:24:05.135049 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:44947
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:46849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:44947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:24:05.255618 11719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:24:05.256027 11719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:24:05.256117 11719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:24:05.260301 11719 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:24:05.260416 11719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:24:05.260455 11719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:24:05.260495 11719 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:24:05.260527 11719 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:24:05.266072 11719 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:46849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:44947
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:44947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:24:05.267305 11719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:24:05.268432 11719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:24:05.277736 11719 server_base.cc:1034] running on GCE node
W20260811 17:24:05.277690 11736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:24:05.277690 11734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:24:05.277684 11731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:24:05.278831 11719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:24:05.280174 11719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:24:05.281383 11719 hybrid_clock.cc:648] HybridClock initialized: now 1786469045281355 us; error 44 us; skew 500 ppm
Aug 11 17:24:05 dist-test-slave-mjjr krb5kdc[11699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469045, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:24:05.284381 11719 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:24:05.285383 11719 webserver.cc:460] Webserver started at http://127.9.157.190:36431/ using document root <none> and password file <none>
I20260811 17:24:05.286067 11719 fs_manager.cc:359] Metadata directory not provided
I20260811 17:24:05.286172 11719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:24:05.286459 11719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:24:05.288872 11719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "0ff3b518e9994f8baf6d06cfd69a1fed"
format_stamp: "Formatted at 2026-08-11 17:24:05 on dist-test-slave-mjjr"
server_key: "934f441cec1b41fcccf8e590c7dcfd08"
server_key_iv: "703707d24d42cea234d045ce66901c30"
server_key_version: "encryptionkey@0"
I20260811 17:24:05.289496 11719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "0ff3b518e9994f8baf6d06cfd69a1fed"
format_stamp: "Formatted at 2026-08-11 17:24:05 on dist-test-slave-mjjr"
server_key: "934f441cec1b41fcccf8e590c7dcfd08"
server_key_iv: "703707d24d42cea234d045ce66901c30"
server_key_version: "encryptionkey@0"
I20260811 17:24:05.293752 11719 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 17:24:05.296434 11748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:24:05.297792 11719 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:24:05.297958 11719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "0ff3b518e9994f8baf6d06cfd69a1fed"
format_stamp: "Formatted at 2026-08-11 17:24:05 on dist-test-slave-mjjr"
server_key: "934f441cec1b41fcccf8e590c7dcfd08"
server_key_iv: "703707d24d42cea234d045ce66901c30"
server_key_version: "encryptionkey@0"
I20260811 17:24:05.298089 11719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:24:05.303174 11719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:24:05.306655 11719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:24:05.306934 11719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:24:05.315037 11719 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:44947
I20260811 17:24:05.315102 11839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:44947 every 8 connection(s)
I20260811 17:24:05.316216 11719 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:24:05.319331 11840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:24:05.323436 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 11719
I20260811 17:24:05.323609 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:24:05.323992 9846 external_mini_cluster.cc:1428] Setting key b9656e36c6316bd6e6d2cfbaedf6d722
I20260811 17:24:05.326238 11840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ff3b518e9994f8baf6d06cfd69a1fed: Bootstrap starting.
I20260811 17:24:05.328953 11840 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ff3b518e9994f8baf6d06cfd69a1fed: Neither blocks nor log segments found. Creating new log.
I20260811 17:24:05.329818 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:05.325588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59609 (local address 127.9.157.190:44947)
0811 17:24:05.326160 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:05.326177 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:05.326231 (+ 54us) server_negotiation.cc:407] Connection header received
0811 17:24:05.327002 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:05.327022 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:05.327335 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:05.327771 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:05.328540 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:05.328545 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:05.328557 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:24:05.328567 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:05.328618 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:24:05.328636 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:05.328772 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:05.328854 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 17:24:05.329026 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":167,"threads_started":1}
I20260811 17:24:05.329823 11840 log.cc:824] T 00000000000000000000000000000000 P 0ff3b518e9994f8baf6d06cfd69a1fed: Log is configured to *not* fsync() on all Append() calls
I20260811 17:24:05.332365 11840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ff3b518e9994f8baf6d06cfd69a1fed: No bootstrap required, opened a new log
I20260811 17:24:05.333043 9846 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:24:05.337080 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:05.335260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46897 (local address 127.9.157.190:44947)
0811 17:24:05.335425 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:05.335432 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:05.335562 (+ 130us) server_negotiation.cc:407] Connection header received
0811 17:24:05.335723 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:05.335728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:05.335777 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:05.335897 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:05.336660 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:05.336664 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:05.336667 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:24:05.336681 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:05.336713 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:24:05.336727 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:05.336823 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:05.336903 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 17:24:05.336944 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 17:24:05.337222 11840 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ff3b518e9994f8baf6d06cfd69a1fed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff3b518e9994f8baf6d06cfd69a1fed" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44947 } }
I20260811 17:24:05.337574 11840 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ff3b518e9994f8baf6d06cfd69a1fed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:24:05.337646 11840 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ff3b518e9994f8baf6d06cfd69a1fed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ff3b518e9994f8baf6d06cfd69a1fed, State: Initialized, Role: FOLLOWER
I20260811 17:24:05.338135 11840 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ff3b518e9994f8baf6d06cfd69a1fed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff3b518e9994f8baf6d06cfd69a1fed" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44947 } }
I20260811 17:24:05.338300 11840 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ff3b518e9994f8baf6d06cfd69a1fed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:24:05.338387 11840 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ff3b518e9994f8baf6d06cfd69a1fed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:24:05.338507 11840 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ff3b518e9994f8baf6d06cfd69a1fed [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:24:05.339622 11840 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ff3b518e9994f8baf6d06cfd69a1fed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff3b518e9994f8baf6d06cfd69a1fed" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44947 } }
I20260811 17:24:05.340076 11840 leader_election.cc:302] T 00000000000000000000000000000000 P 0ff3b518e9994f8baf6d06cfd69a1fed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ff3b518e9994f8baf6d06cfd69a1fed; no voters:
I20260811 17:24:05.340492 11840 leader_election.cc:288] T 00000000000000000000000000000000 P 0ff3b518e9994f8baf6d06cfd69a1fed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:24:05.340679 11852 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ff3b518e9994f8baf6d06cfd69a1fed [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:24:05.341003 11852 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ff3b518e9994f8baf6d06cfd69a1fed [term 1 LEADER]: Becoming Leader. State: Replica: 0ff3b518e9994f8baf6d06cfd69a1fed, State: Running, Role: LEADER
I20260811 17:24:05.341348 11852 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ff3b518e9994f8baf6d06cfd69a1fed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff3b518e9994f8baf6d06cfd69a1fed" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44947 } }
I20260811 17:24:05.341969 11840 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ff3b518e9994f8baf6d06cfd69a1fed [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:24:05.343532 11853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ff3b518e9994f8baf6d06cfd69a1fed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ff3b518e9994f8baf6d06cfd69a1fed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff3b518e9994f8baf6d06cfd69a1fed" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44947 } } }
I20260811 17:24:05.343894 11854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ff3b518e9994f8baf6d06cfd69a1fed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ff3b518e9994f8baf6d06cfd69a1fed. Latest consensus state: current_term: 1 leader_uuid: "0ff3b518e9994f8baf6d06cfd69a1fed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff3b518e9994f8baf6d06cfd69a1fed" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 44947 } } }
I20260811 17:24:05.344023 11853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ff3b518e9994f8baf6d06cfd69a1fed [sys.catalog]: This master's current role is: LEADER
I20260811 17:24:05.344056 11854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ff3b518e9994f8baf6d06cfd69a1fed [sys.catalog]: This master's current role is: LEADER
W20260811 17:24:05.347429 11875 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0ff3b518e9994f8baf6d06cfd69a1fed: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:24:05.347610 11875 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:24:05.347791 11876 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:24:05.348845 11876 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:24:05.355353 11876 catalog_manager.cc:1347] Generated new cluster ID: 6fb31d4a982243418e5aa1b4f33dfe77
I20260811 17:24:05.355446 11876 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:24:05.369223 11876 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:24:05.369625 11876 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:24:05.374503 11876 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ff3b518e9994f8baf6d06cfd69a1fed: Generated new TSK 0
I20260811 17:24:05.375375 11876 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:24:05.385550 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 11719
2026-08-11T17:24:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1676 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:24:05 dist-test-slave-mjjr krb5kdc[11886](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:24:05 dist-test-slave-mjjr krb5kdc[11886](info): set up 2 sockets
Aug 11 17:24:05 dist-test-slave-mjjr krb5kdc[11886](info): commencing operation
krb5kdc: starting...
W20260811 17:24:06.674531 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:24:06 dist-test-slave-mjjr krb5kdc[11886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:24:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:24:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:24:06.814833 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:41381
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:42703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:41381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:24:06.937105 11907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:24:06.937494 11907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:24:06.937583 11907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:24:06.941895 11907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:24:06.942022 11907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:24:06.942063 11907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:24:06.942117 11907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:24:06.942173 11907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:24:06.948065 11907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:42703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:41381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:41381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:24:06.949370 11907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:24:06.950539 11907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:24:06.960635 11919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:24:06.960616 11916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:24:06.960883 11907 server_base.cc:1034] running on GCE node
W20260811 17:24:06.960616 11917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:24:06.961750 11907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:24:06.963085 11907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:24:06.964293 11907 hybrid_clock.cc:648] HybridClock initialized: now 1786469046964262 us; error 40 us; skew 500 ppm
Aug 11 17:24:06 dist-test-slave-mjjr krb5kdc[11886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469046, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:24:06.971632 11907 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:24:06.972713 11907 webserver.cc:460] Webserver started at http://127.9.157.190:46447/ using document root <none> and password file <none>
I20260811 17:24:06.973345 11907 fs_manager.cc:359] Metadata directory not provided
I20260811 17:24:06.973448 11907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:24:06.973703 11907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:24:06.975971 11907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "d8eb3e4ff5644a2e9bfb0eb7633165a5"
format_stamp: "Formatted at 2026-08-11 17:24:06 on dist-test-slave-mjjr"
server_key: "0eb3392623425dfd2b34f77d8bb22ecd"
server_key_iv: "3e7531c9eab4b66a27ec7a77d3f86595"
server_key_version: "encryptionkey@0"
I20260811 17:24:06.976559 11907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "d8eb3e4ff5644a2e9bfb0eb7633165a5"
format_stamp: "Formatted at 2026-08-11 17:24:06 on dist-test-slave-mjjr"
server_key: "0eb3392623425dfd2b34f77d8bb22ecd"
server_key_iv: "3e7531c9eab4b66a27ec7a77d3f86595"
server_key_version: "encryptionkey@0"
I20260811 17:24:06.980499 11907 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 17:24:06.983084 11932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:24:06.984393 11907 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 17:24:06.984560 11907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "d8eb3e4ff5644a2e9bfb0eb7633165a5"
format_stamp: "Formatted at 2026-08-11 17:24:06 on dist-test-slave-mjjr"
server_key: "0eb3392623425dfd2b34f77d8bb22ecd"
server_key_iv: "3e7531c9eab4b66a27ec7a77d3f86595"
server_key_version: "encryptionkey@0"
I20260811 17:24:06.984684 11907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:24:07.023402 11907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:24:07.026825 11907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:24:07.027077 11907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:24:07.034987 11907 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:41381
I20260811 17:24:07.035013 12023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:41381 every 8 connection(s)
I20260811 17:24:07.036120 11907 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:24:07.039144 12024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:24:07.044250 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 11907
I20260811 17:24:07.044375 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:24:07.044653 9846 external_mini_cluster.cc:1428] Setting key 2499130c096877d7011edd57a19804e7
I20260811 17:24:07.045674 12024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8eb3e4ff5644a2e9bfb0eb7633165a5: Bootstrap starting.
I20260811 17:24:07.048393 12024 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8eb3e4ff5644a2e9bfb0eb7633165a5: Neither blocks nor log segments found. Creating new log.
I20260811 17:24:07.049247 12024 log.cc:824] T 00000000000000000000000000000000 P d8eb3e4ff5644a2e9bfb0eb7633165a5: Log is configured to *not* fsync() on all Append() calls
I20260811 17:24:07.051360 12024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8eb3e4ff5644a2e9bfb0eb7633165a5: No bootstrap required, opened a new log
Aug 11 17:24:07 dist-test-slave-mjjr krb5kdc[11886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:24:07.056744 12024 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8eb3e4ff5644a2e9bfb0eb7633165a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8eb3e4ff5644a2e9bfb0eb7633165a5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41381 } }
I20260811 17:24:07.057070 12024 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8eb3e4ff5644a2e9bfb0eb7633165a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:24:07.057214 12024 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8eb3e4ff5644a2e9bfb0eb7633165a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8eb3e4ff5644a2e9bfb0eb7633165a5, State: Initialized, Role: FOLLOWER
I20260811 17:24:07.057770 12024 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8eb3e4ff5644a2e9bfb0eb7633165a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8eb3e4ff5644a2e9bfb0eb7633165a5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41381 } }
I20260811 17:24:07.057940 12024 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8eb3e4ff5644a2e9bfb0eb7633165a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:24:07.058040 12024 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8eb3e4ff5644a2e9bfb0eb7633165a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:24:07.058156 12024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8eb3e4ff5644a2e9bfb0eb7633165a5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:24:07.059218 12024 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8eb3e4ff5644a2e9bfb0eb7633165a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8eb3e4ff5644a2e9bfb0eb7633165a5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41381 } }
I20260811 17:24:07.059615 12024 leader_election.cc:302] T 00000000000000000000000000000000 P d8eb3e4ff5644a2e9bfb0eb7633165a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8eb3e4ff5644a2e9bfb0eb7633165a5; no voters:
I20260811 17:24:07.060066 12024 leader_election.cc:288] T 00000000000000000000000000000000 P d8eb3e4ff5644a2e9bfb0eb7633165a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:24:07.060184 12034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8eb3e4ff5644a2e9bfb0eb7633165a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:24:07.060422 12028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:07.046191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51311 (local address 127.9.157.190:41381)
0811 17:24:07.046696 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:07.046711 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:07.046748 (+ 37us) server_negotiation.cc:407] Connection header received
0811 17:24:07.047404 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:07.047421 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:07.047855 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:07.048196 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:07.049037 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:07.050065 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:07.050936 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:07.051414 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:07.051590 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:07.054665 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:07.054675 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:07.054712 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:07.054756 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:07.057013 (+ 2257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:07.057593 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:07.057599 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:07.057604 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:07.057720 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:07.058031 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:07.058037 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:07.058041 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:07.058549 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:07.058772 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:07.059047 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 17:24:07.059401 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":176,"threads_started":1}
I20260811 17:24:07.060410 12034 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8eb3e4ff5644a2e9bfb0eb7633165a5 [term 1 LEADER]: Becoming Leader. State: Replica: d8eb3e4ff5644a2e9bfb0eb7633165a5, State: Running, Role: LEADER
I20260811 17:24:07.060778 12034 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8eb3e4ff5644a2e9bfb0eb7633165a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8eb3e4ff5644a2e9bfb0eb7633165a5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41381 } }
I20260811 17:24:07.061352 12024 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8eb3e4ff5644a2e9bfb0eb7633165a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:24:07.062026 12034 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8eb3e4ff5644a2e9bfb0eb7633165a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8eb3e4ff5644a2e9bfb0eb7633165a5. Latest consensus state: current_term: 1 leader_uuid: "d8eb3e4ff5644a2e9bfb0eb7633165a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8eb3e4ff5644a2e9bfb0eb7633165a5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41381 } } }
I20260811 17:24:07.062171 12034 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8eb3e4ff5644a2e9bfb0eb7633165a5 [sys.catalog]: This master's current role is: LEADER
I20260811 17:24:07.062748 12035 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8eb3e4ff5644a2e9bfb0eb7633165a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8eb3e4ff5644a2e9bfb0eb7633165a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8eb3e4ff5644a2e9bfb0eb7633165a5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 41381 } } }
I20260811 17:24:07.063097 12035 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8eb3e4ff5644a2e9bfb0eb7633165a5 [sys.catalog]: This master's current role is: LEADER
I20260811 17:24:07.063186 12045 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:24:07.064144 9846 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:24:07.066725 12045 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:24:07.074517 12045 catalog_manager.cc:1347] Generated new cluster ID: 0392146812624e16b2d8a39f098a73f0
I20260811 17:24:07.074599 12045 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:24:07.076426 12028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:07.066770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49899 (local address 127.9.157.190:41381)
0811 17:24:07.066902 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:07.066907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:07.067110 (+ 203us) server_negotiation.cc:407] Connection header received
0811 17:24:07.067376 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:07.067383 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:07.067446 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:07.067549 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:07.068447 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:07.069002 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:07.070125 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:07.070461 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:07.070575 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:07.072755 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:07.072758 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:07.072760 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:07.072785 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:07.074291 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:07.074794 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:07.074798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:07.074800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:07.074890 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:07.075187 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:07.075191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:07.075193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:07.075390 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:07.075526 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:07.076176 (+ 650us) server_negotiation.cc:299] Negotiation successful
0811 17:24:07.076297 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 17:24:07.093492 12045 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:24:07.094560 12045 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:24:07.102663 12045 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8eb3e4ff5644a2e9bfb0eb7633165a5: Generated new TSK 0
I20260811 17:24:07.103572 12045 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:24:07.113291 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 11907
2026-08-11T17:24:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1720 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:24:07 dist-test-slave-mjjr krb5kdc[12071](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:24:07 dist-test-slave-mjjr krb5kdc[12071](info): set up 2 sockets
Aug 11 17:24:07 dist-test-slave-mjjr krb5kdc[12071](info): commencing operation
krb5kdc: starting...
W20260811 17:24:08.375423 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:24:08 dist-test-slave-mjjr krb5kdc[12071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:24:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:24:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:24:08.519464 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:43231
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:46647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:43231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:24:08.642402 12100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:24:08.642696 12100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:24:08.642781 12100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:24:08.646924 12100 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:24:08.647001 12100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:24:08.647022 12100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:24:08.647038 12100 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:24:08.647054 12100 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:24:08.652670 12100 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:46647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:43231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:43231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:24:08.653882 12100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:24:08.655032 12100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:24:08.664219 12110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:24:08.664219 12113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:24:08.664224 12109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:24:08.664526 12100 server_base.cc:1034] running on GCE node
I20260811 17:24:08.665158 12100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:24:08.666393 12100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:24:08.667682 12100 hybrid_clock.cc:648] HybridClock initialized: now 1786469048667617 us; error 35 us; skew 500 ppm
Aug 11 17:24:08 dist-test-slave-mjjr krb5kdc[12071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469048, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:24:08.670676 12100 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:24:08.671833 12100 webserver.cc:460] Webserver started at http://127.9.157.190:35679/ using document root <none> and password file <none>
I20260811 17:24:08.672482 12100 fs_manager.cc:359] Metadata directory not provided
I20260811 17:24:08.672585 12100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:24:08.672847 12100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:24:08.675127 12100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "31af7c67f25f4f7faa863b8fba062fee"
format_stamp: "Formatted at 2026-08-11 17:24:08 on dist-test-slave-mjjr"
server_key: "5c95a8382da5e7a2532d5c22aa4f6739"
server_key_iv: "48a7891825674b967e5074c0c00cd26b"
server_key_version: "encryptionkey@0"
I20260811 17:24:08.675804 12100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "31af7c67f25f4f7faa863b8fba062fee"
format_stamp: "Formatted at 2026-08-11 17:24:08 on dist-test-slave-mjjr"
server_key: "5c95a8382da5e7a2532d5c22aa4f6739"
server_key_iv: "48a7891825674b967e5074c0c00cd26b"
server_key_version: "encryptionkey@0"
I20260811 17:24:08.679970 12100 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:24:08.682812 12127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:24:08.684239 12100 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:24:08.684414 12100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "31af7c67f25f4f7faa863b8fba062fee"
format_stamp: "Formatted at 2026-08-11 17:24:08 on dist-test-slave-mjjr"
server_key: "5c95a8382da5e7a2532d5c22aa4f6739"
server_key_iv: "48a7891825674b967e5074c0c00cd26b"
server_key_version: "encryptionkey@0"
I20260811 17:24:08.684551 12100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:24:08.712697 12100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:24:08.716106 12100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:24:08.716391 12100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:24:08.724951 12100 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:43231
I20260811 17:24:08.724985 12209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:43231 every 8 connection(s)
I20260811 17:24:08.726084 12100 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:24:08.728299 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 12100
I20260811 17:24:08.728456 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:24:08.728775 9846 external_mini_cluster.cc:1428] Setting key 76bf8212078fcd887907760880654d13
I20260811 17:24:08.729521 12210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:24:08.736958 12210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31af7c67f25f4f7faa863b8fba062fee: Bootstrap starting.
I20260811 17:24:08.738688 12213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:08.730392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36681 (local address 127.9.157.190:43231)
0811 17:24:08.730879 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:08.730918 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:08.730956 (+ 38us) server_negotiation.cc:407] Connection header received
0811 17:24:08.731633 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:08.731719 (+ 86us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:08.732098 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:08.732390 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:08.732814 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:08.733854 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:08.735115 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:08.735622 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:08.735830 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:08.736779 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:08.736785 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:08.736799 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:24:08.737290 (+ 491us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:08.737353 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:24:08.737379 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:08.737467 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:08.737636 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 17:24:08.737913 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":178,"threads_started":1}
I20260811 17:24:08.739542 12210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31af7c67f25f4f7faa863b8fba062fee: Neither blocks nor log segments found. Creating new log.
I20260811 17:24:08.740520 12210 log.cc:824] T 00000000000000000000000000000000 P 31af7c67f25f4f7faa863b8fba062fee: Log is configured to *not* fsync() on all Append() calls
I20260811 17:24:08.741979 9846 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:24:08.742812 12210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31af7c67f25f4f7faa863b8fba062fee: No bootstrap required, opened a new log
I20260811 17:24:08.746829 12210 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31af7c67f25f4f7faa863b8fba062fee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31af7c67f25f4f7faa863b8fba062fee" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43231 } }
I20260811 17:24:08.747162 12210 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31af7c67f25f4f7faa863b8fba062fee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:24:08.747228 12210 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31af7c67f25f4f7faa863b8fba062fee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31af7c67f25f4f7faa863b8fba062fee, State: Initialized, Role: FOLLOWER
I20260811 17:24:08.747859 12210 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31af7c67f25f4f7faa863b8fba062fee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31af7c67f25f4f7faa863b8fba062fee" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43231 } }
I20260811 17:24:08.748055 12210 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31af7c67f25f4f7faa863b8fba062fee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:24:08.748142 12210 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31af7c67f25f4f7faa863b8fba062fee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:24:08.748276 12210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31af7c67f25f4f7faa863b8fba062fee [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:24:08.749167 12213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:08.744395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51545 (local address 127.9.157.190:43231)
0811 17:24:08.744543 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:08.744548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:08.744728 (+ 180us) server_negotiation.cc:407] Connection header received
0811 17:24:08.744852 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:08.744854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:08.744897 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:08.744968 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:08.745283 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:08.746162 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:08.747305 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:08.747584 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:08.747799 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:08.748691 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:08.748694 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:08.748696 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:24:08.748708 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:08.748732 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:24:08.748743 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:08.748828 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:08.748963 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 17:24:08.749012 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 17:24:08.749631 12210 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31af7c67f25f4f7faa863b8fba062fee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31af7c67f25f4f7faa863b8fba062fee" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43231 } }
I20260811 17:24:08.750015 12210 leader_election.cc:302] T 00000000000000000000000000000000 P 31af7c67f25f4f7faa863b8fba062fee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31af7c67f25f4f7faa863b8fba062fee; no voters:
I20260811 17:24:08.750361 12210 leader_election.cc:288] T 00000000000000000000000000000000 P 31af7c67f25f4f7faa863b8fba062fee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:24:08.750479 12229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31af7c67f25f4f7faa863b8fba062fee [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:24:08.750772 12229 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31af7c67f25f4f7faa863b8fba062fee [term 1 LEADER]: Becoming Leader. State: Replica: 31af7c67f25f4f7faa863b8fba062fee, State: Running, Role: LEADER
I20260811 17:24:08.751144 12229 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31af7c67f25f4f7faa863b8fba062fee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31af7c67f25f4f7faa863b8fba062fee" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43231 } }
I20260811 17:24:08.751737 12210 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31af7c67f25f4f7faa863b8fba062fee [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:24:08.753000 12235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31af7c67f25f4f7faa863b8fba062fee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31af7c67f25f4f7faa863b8fba062fee. Latest consensus state: current_term: 1 leader_uuid: "31af7c67f25f4f7faa863b8fba062fee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31af7c67f25f4f7faa863b8fba062fee" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43231 } } }
I20260811 17:24:08.753002 12232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31af7c67f25f4f7faa863b8fba062fee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31af7c67f25f4f7faa863b8fba062fee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31af7c67f25f4f7faa863b8fba062fee" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 43231 } } }
I20260811 17:24:08.753225 12235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31af7c67f25f4f7faa863b8fba062fee [sys.catalog]: This master's current role is: LEADER
I20260811 17:24:08.753225 12232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31af7c67f25f4f7faa863b8fba062fee [sys.catalog]: This master's current role is: LEADER
I20260811 17:24:08.753702 12254 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:24:08.757081 12254 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:24:08.763212 12254 catalog_manager.cc:1347] Generated new cluster ID: 13d75758429e4f7e9eb6d3cf418528b9
I20260811 17:24:08.763317 12254 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:24:08.770998 12254 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:24:08.772395 12254 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:24:08.798295 12254 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31af7c67f25f4f7faa863b8fba062fee: Generated new TSK 0
I20260811 17:24:08.799245 12254 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:24:08.808836 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 12100
2026-08-11T17:24:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1701 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:24:08 dist-test-slave-mjjr krb5kdc[12274](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:24:08 dist-test-slave-mjjr krb5kdc[12274](info): set up 2 sockets
Aug 11 17:24:08 dist-test-slave-mjjr krb5kdc[12274](info): commencing operation
krb5kdc: starting...
W20260811 17:24:10.138217 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.295s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:24:10 dist-test-slave-mjjr krb5kdc[12274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:24:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:24:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:24:10.273224 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:37185
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:43479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:37185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:24:10.393319 12297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:24:10.393615 12297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:24:10.393659 12297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:24:10.397915 12297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:24:10.398000 12297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:24:10.398020 12297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:24:10.398036 12297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:24:10.398051 12297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:24:10.403825 12297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:43479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:37185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:37185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:24:10.405048 12297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:24:10.406303 12297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 17:24:10.415983 12297 server_base.cc:1034] running on GCE node
W20260811 17:24:10.415889 12309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:24:10.415930 12307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:24:10.415908 12312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:24:10.416935 12297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:24:10.418216 12297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:24:10.419430 12297 hybrid_clock.cc:648] HybridClock initialized: now 1786469050419412 us; error 35 us; skew 500 ppm
Aug 11 17:24:10 dist-test-slave-mjjr krb5kdc[12274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469050, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:24:10.422362 12297 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:24:10.423390 12297 webserver.cc:460] Webserver started at http://127.9.157.190:35125/ using document root <none> and password file <none>
I20260811 17:24:10.424072 12297 fs_manager.cc:359] Metadata directory not provided
I20260811 17:24:10.424172 12297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:24:10.424433 12297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:24:10.426602 12297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "f3f9bfdd5e55434e872cb5e773836be5"
format_stamp: "Formatted at 2026-08-11 17:24:10 on dist-test-slave-mjjr"
server_key: "ab7efd1219f234de720633c230cbb090"
server_key_iv: "59d37680323865567f975f33d0484cb9"
server_key_version: "encryptionkey@0"
I20260811 17:24:10.427197 12297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "f3f9bfdd5e55434e872cb5e773836be5"
format_stamp: "Formatted at 2026-08-11 17:24:10 on dist-test-slave-mjjr"
server_key: "ab7efd1219f234de720633c230cbb090"
server_key_iv: "59d37680323865567f975f33d0484cb9"
server_key_version: "encryptionkey@0"
I20260811 17:24:10.431284 12297 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 17:24:10.434120 12323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:24:10.435410 12297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:24:10.435573 12297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "f3f9bfdd5e55434e872cb5e773836be5"
format_stamp: "Formatted at 2026-08-11 17:24:10 on dist-test-slave-mjjr"
server_key: "ab7efd1219f234de720633c230cbb090"
server_key_iv: "59d37680323865567f975f33d0484cb9"
server_key_version: "encryptionkey@0"
I20260811 17:24:10.435752 12297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:24:10.441356 12297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:24:10.444810 12297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:24:10.445050 12297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:24:10.452788 12297 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:37185
I20260811 17:24:10.452816 12413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:37185 every 8 connection(s)
I20260811 17:24:10.454077 12297 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:24:10.457059 12414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:24:10.461694 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 12297
I20260811 17:24:10.461844 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:24:10.462160 9846 external_mini_cluster.cc:1428] Setting key 8154d73833d81ef4582c19e81ae19aba
I20260811 17:24:10.463709 12414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3f9bfdd5e55434e872cb5e773836be5: Bootstrap starting.
I20260811 17:24:10.466462 12414 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3f9bfdd5e55434e872cb5e773836be5: Neither blocks nor log segments found. Creating new log.
I20260811 17:24:10.467342 12414 log.cc:824] T 00000000000000000000000000000000 P f3f9bfdd5e55434e872cb5e773836be5: Log is configured to *not* fsync() on all Append() calls
I20260811 17:24:10.468446 12421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:10.463811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56895 (local address 127.9.157.190:37185)
0811 17:24:10.464357 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:10.464379 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:10.464424 (+ 45us) server_negotiation.cc:407] Connection header received
0811 17:24:10.465241 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:10.465263 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:10.465585 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:10.465957 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:10.466912 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:10.466917 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:10.466930 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:24:10.466941 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:10.466996 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:24:10.467017 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:10.467143 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:10.467253 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 17:24:10.467435 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":158,"threads_started":1}
I20260811 17:24:10.469743 12414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3f9bfdd5e55434e872cb5e773836be5: No bootstrap required, opened a new log
I20260811 17:24:10.471689 9846 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:24:10.474484 12414 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3f9bfdd5e55434e872cb5e773836be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3f9bfdd5e55434e872cb5e773836be5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37185 } }
I20260811 17:24:10.474782 12414 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3f9bfdd5e55434e872cb5e773836be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 17:24:10.474871 12414 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3f9bfdd5e55434e872cb5e773836be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3f9bfdd5e55434e872cb5e773836be5, State: Initialized, Role: FOLLOWER
I20260811 17:24:10.475415 12414 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3f9bfdd5e55434e872cb5e773836be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3f9bfdd5e55434e872cb5e773836be5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37185 } }
I20260811 17:24:10.475611 12414 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3f9bfdd5e55434e872cb5e773836be5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 17:24:10.475776 12414 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3f9bfdd5e55434e872cb5e773836be5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 17:24:10.475921 12414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3f9bfdd5e55434e872cb5e773836be5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:24:10.476024 12421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:10.474027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59567 (local address 127.9.157.190:37185)
0811 17:24:10.474271 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:10.474278 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:10.474298 (+ 20us) server_negotiation.cc:407] Connection header received
0811 17:24:10.474474 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:10.474478 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:10.474524 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:10.474609 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:10.475427 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:10.475431 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:10.475434 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 17:24:10.475446 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:10.475484 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 17:24:10.475496 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:10.475591 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:10.475718 (+ 127us) server_negotiation.cc:299] Negotiation successful
0811 17:24:10.475766 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
I20260811 17:24:10.477314 12414 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3f9bfdd5e55434e872cb5e773836be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3f9bfdd5e55434e872cb5e773836be5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37185 } }
I20260811 17:24:10.477743 12414 leader_election.cc:302] T 00000000000000000000000000000000 P f3f9bfdd5e55434e872cb5e773836be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3f9bfdd5e55434e872cb5e773836be5; no voters:
I20260811 17:24:10.478287 12414 leader_election.cc:288] T 00000000000000000000000000000000 P f3f9bfdd5e55434e872cb5e773836be5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 17:24:10.478497 12435 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3f9bfdd5e55434e872cb5e773836be5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:24:10.478756 12435 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3f9bfdd5e55434e872cb5e773836be5 [term 1 LEADER]: Becoming Leader. State: Replica: f3f9bfdd5e55434e872cb5e773836be5, State: Running, Role: LEADER
I20260811 17:24:10.479177 12435 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3f9bfdd5e55434e872cb5e773836be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3f9bfdd5e55434e872cb5e773836be5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37185 } }
I20260811 17:24:10.479902 12414 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3f9bfdd5e55434e872cb5e773836be5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:24:10.481099 12437 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3f9bfdd5e55434e872cb5e773836be5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3f9bfdd5e55434e872cb5e773836be5. Latest consensus state: current_term: 1 leader_uuid: "f3f9bfdd5e55434e872cb5e773836be5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3f9bfdd5e55434e872cb5e773836be5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37185 } } }
I20260811 17:24:10.481249 12437 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3f9bfdd5e55434e872cb5e773836be5 [sys.catalog]: This master's current role is: LEADER
I20260811 17:24:10.481665 12452 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:24:10.481628 12436 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3f9bfdd5e55434e872cb5e773836be5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3f9bfdd5e55434e872cb5e773836be5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3f9bfdd5e55434e872cb5e773836be5" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 37185 } } }
I20260811 17:24:10.481776 12436 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3f9bfdd5e55434e872cb5e773836be5 [sys.catalog]: This master's current role is: LEADER
I20260811 17:24:10.485214 12452 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:24:10.491926 12452 catalog_manager.cc:1347] Generated new cluster ID: 5ddb47c11d684140b5a7380103a0b74b
I20260811 17:24:10.492024 12452 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:24:10.504873 12452 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:24:10.505265 12452 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:24:10.513152 12452 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3f9bfdd5e55434e872cb5e773836be5: Generated new TSK 0
I20260811 17:24:10.514104 12452 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 17:24:10.523109 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 12297
2026-08-11T17:24:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1713 ms)
[----------] 18 tests from AuthTokenIssuingTest (30736 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:24:10 dist-test-slave-mjjr krb5kdc[12472](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:24:10 dist-test-slave-mjjr krb5kdc[12472](info): set up 2 sockets
Aug 11 17:24:10 dist-test-slave-mjjr krb5kdc[12472](info): commencing operation
krb5kdc: starting...
W20260811 17:24:11.801591 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:24:11 dist-test-slave-mjjr krb5kdc[12472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469051, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:24:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:24:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:24:11.940768 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:45409
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:44059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:45409,127.9.157.189:37241,127.9.157.188:45039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:24:12.057834 12495 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 17:24:12.058223 12495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:24:12.058305 12495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:24:12.058383 12495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:24:12.062543 12495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:24:12.062661 12495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:24:12.062700 12495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:24:12.062740 12495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:24:12.062773 12495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:24:12.068434 12495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:44059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:45409,127.9.157.189:37241,127.9.157.188:45039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:45409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:24:12.069736 12495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:24:12.070825 12495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:24:12.079694 12506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:24:12.079716 12508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:24:12.079717 12505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:24:12.079967 12495 server_base.cc:1034] running on GCE node
I20260811 17:24:12.080716 12495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:24:12.081887 12495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:24:12.083055 12495 hybrid_clock.cc:648] HybridClock initialized: now 1786469052083025 us; error 43 us; skew 500 ppm
Aug 11 17:24:12 dist-test-slave-mjjr krb5kdc[12472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469052, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:24:12.090135 12495 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:24:12.091209 12495 webserver.cc:460] Webserver started at http://127.9.157.190:40745/ using document root <none> and password file <none>
I20260811 17:24:12.091924 12495 fs_manager.cc:359] Metadata directory not provided
I20260811 17:24:12.092026 12495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:24:12.092280 12495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:24:12.094553 12495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "682ca2251d7449118763f40113fd3fd4"
format_stamp: "Formatted at 2026-08-11 17:24:12 on dist-test-slave-mjjr"
server_key: "3b833765a9b0cb5f5710072bda125aa3"
server_key_iv: "8efa7596ab65c699f8f2a5335b8deba2"
server_key_version: "encryptionkey@0"
I20260811 17:24:12.095188 12495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "682ca2251d7449118763f40113fd3fd4"
format_stamp: "Formatted at 2026-08-11 17:24:12 on dist-test-slave-mjjr"
server_key: "3b833765a9b0cb5f5710072bda125aa3"
server_key_iv: "8efa7596ab65c699f8f2a5335b8deba2"
server_key_version: "encryptionkey@0"
I20260811 17:24:12.099557 12495 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 17:24:12.102279 12517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:24:12.103724 12495 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 17:24:12.103902 12495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "682ca2251d7449118763f40113fd3fd4"
format_stamp: "Formatted at 2026-08-11 17:24:12 on dist-test-slave-mjjr"
server_key: "3b833765a9b0cb5f5710072bda125aa3"
server_key_iv: "8efa7596ab65c699f8f2a5335b8deba2"
server_key_version: "encryptionkey@0"
I20260811 17:24:12.104034 12495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:24:12.117825 12495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:24:12.121160 12495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:24:12.121392 12495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:24:12.129072 12495 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:45409
I20260811 17:24:12.129428 12602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:45409 every 8 connection(s)
I20260811 17:24:12.130206 12495 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:24:12.133304 12603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:24:12.137581 12603 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.190" port: 45409 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:24:12.139568 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 12495
I20260811 17:24:12.139741 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:24:12.140071 9846 external_mini_cluster.cc:1428] Setting key 11a91d4f839ae1757d3a2d01f0387089
Aug 11 17:24:12 dist-test-slave-mjjr krb5kdc[12472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469052, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.157.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.189@KRBTEST.COM" created.
I20260811 17:24:12.154381 12605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.140294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:45409 (local address 127.0.0.1:39696)
0811 17:24:12.140914 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.140963 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 17:24:12.142092 (+ 1129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:24:12.142833 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:24:12.142838 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:24:12.143329 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:24:12.143593 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:12.143608 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.144847 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.144852 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:12.145633 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:12.145642 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.146343 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.146347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:12.146463 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:12.147193 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:24:12.147211 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:24:12.148839 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:24:12.151152 (+ 2313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:12.151163 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:12.151175 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:12.151430 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:12.151923 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:12.151926 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:12.151928 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:12.152060 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:12.152876 (+ 816us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:24:12.152887 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:24:12.153056 (+ 169us) client_negotiation.cc:769] Sending connection context
0811 17:24:12.153209 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 17:24:12.153512 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":176,"threads_started":1}
I20260811 17:24:12.154385 12607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.141360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39696 (local address 127.9.157.190:45409)
0811 17:24:12.141848 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:12.141862 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:12.141912 (+ 50us) server_negotiation.cc:407] Connection header received
0811 17:24:12.142441 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:12.142446 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:12.142637 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:12.142841 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:12.143822 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.144704 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.145768 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.146163 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.146347 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:12.149009 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:12.149016 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:12.149027 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:12.149050 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:12.151029 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.151564 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.151569 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.151571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.151749 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.152175 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.152178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.152179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.152727 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:12.152871 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:12.153404 (+ 533us) server_negotiation.cc:299] Negotiation successful
0811 17:24:12.153592 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":341,"thread_start_us":98,"threads_started":1}
I20260811 17:24:12.157337 12603 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.189" port: 37241 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:24:12.158362 12605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.157961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:37241 (local address 127.0.0.1:46966)
0811 17:24:12.158137 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.158247 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.189:37241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 17:24:12.158661 12522 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.157.189:37241: connect: Connection refused (error 111)
I20260811 17:24:12.159734 12605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.159385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:37241 (local address 127.0.0.1:46978)
0811 17:24:12.159533 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.159605 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.189:37241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 17:24:12.159955 12603 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.189:37241: Network error: Client connection negotiation failed: client connection to 127.9.157.189:37241: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.189.keytab.
Entry for principal kudu/127.9.157.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.189.keytab.
I20260811 17:24:12.171384 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.189:37241
--webserver_interface=127.9.157.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.189.keytab
--principal=kudu/127.9.157.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:44059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:45409,127.9.157.189:37241,127.9.157.188:45039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:24:12.209256 12603 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.189" port: 37241 } attempt: 1
I20260811 17:24:12.210206 12605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.209847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:37241 (local address 127.0.0.1:46984)
0811 17:24:12.210026 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.210109 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.189:37241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 17:24:12.211071 12605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.210827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:37241 (local address 127.0.0.1:46998)
0811 17:24:12.210948 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.211003 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.189:37241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 17:24:12.211225 12603 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.189:37241: Network error: Client connection negotiation failed: client connection to 127.9.157.189:37241: connect: Connection refused (error 111)
I20260811 17:24:12.279448 12603 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.189" port: 37241 } attempt: 2
I20260811 17:24:12.280408 12605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.280044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:37241 (local address 127.0.0.1:47014)
0811 17:24:12.280241 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.280316 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.189:37241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 17:24:12.281239 12605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.280989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:37241 (local address 127.0.0.1:47018)
0811 17:24:12.281112 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.281165 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.189:37241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 17:24:12.281387 12603 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.189:37241: Network error: Client connection negotiation failed: client connection to 127.9.157.189:37241: connect: Connection refused (error 111)
W20260811 17:24:12.291800 12613 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 17:24:12.292133 12613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:24:12.292182 12613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:24:12.292268 12613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:24:12.296460 12613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:24:12.296527 12613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:24:12.296547 12613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:24:12.296605 12613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:24:12.296636 12613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:24:12.302167 12613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:44059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:45409,127.9.157.189:37241,127.9.157.188:45039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.189.keytab
--principal=kudu/127.9.157.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.189:37241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.157.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:24:12.303369 12613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:24:12.304528 12613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:24:12.314455 12620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:24:12.314476 12623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:24:12.314469 12626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:24:12.314823 12613 server_base.cc:1034] running on GCE node
I20260811 17:24:12.315565 12613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:24:12.316957 12613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:24:12.318163 12613 hybrid_clock.cc:648] HybridClock initialized: now 1786469052318144 us; error 37 us; skew 500 ppm
Aug 11 17:24:12 dist-test-slave-mjjr krb5kdc[12472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469052, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:24:12.321441 12613 init.cc:377] Logged in from keytab as kudu/127.9.157.189@KRBTEST.COM (short username kudu)
I20260811 17:24:12.322535 12613 webserver.cc:460] Webserver started at http://127.9.157.189:34357/ using document root <none> and password file <none>
I20260811 17:24:12.323182 12613 fs_manager.cc:359] Metadata directory not provided
I20260811 17:24:12.323251 12613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:24:12.323520 12613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:24:12.325832 12613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-1/data/instance:
uuid: "935ef3abfeee42fe83b0fccc2d4dad96"
format_stamp: "Formatted at 2026-08-11 17:24:12 on dist-test-slave-mjjr"
server_key: "80a0611c3bfded1a89eaf44cf0250dd1"
server_key_iv: "afe30ead7f187cf8aea9fb0b2f8bcbc9"
server_key_version: "encryptionkey@0"
I20260811 17:24:12.326516 12613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-1/wal/instance:
uuid: "935ef3abfeee42fe83b0fccc2d4dad96"
format_stamp: "Formatted at 2026-08-11 17:24:12 on dist-test-slave-mjjr"
server_key: "80a0611c3bfded1a89eaf44cf0250dd1"
server_key_iv: "afe30ead7f187cf8aea9fb0b2f8bcbc9"
server_key_version: "encryptionkey@0"
I20260811 17:24:12.330822 12613 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 17:24:12.333602 12637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:24:12.334949 12613 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 17:24:12.335125 12613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-1/wal
uuid: "935ef3abfeee42fe83b0fccc2d4dad96"
format_stamp: "Formatted at 2026-08-11 17:24:12 on dist-test-slave-mjjr"
server_key: "80a0611c3bfded1a89eaf44cf0250dd1"
server_key_iv: "afe30ead7f187cf8aea9fb0b2f8bcbc9"
server_key_version: "encryptionkey@0"
I20260811 17:24:12.335258 12613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:24:12.346275 12613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:24:12.349709 12613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:24:12.349946 12613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:24:12.357656 12613 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.189:37241
I20260811 17:24:12.357982 12714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.189:37241 every 8 connection(s)
I20260811 17:24:12.358721 12613 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-1/data/info.pb
I20260811 17:24:12.359560 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 12613
I20260811 17:24:12.359743 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-1/wal/instance
I20260811 17:24:12.360054 9846 external_mini_cluster.cc:1428] Setting key aa8a4b3611d7c730a3c0de66da0f27fb
I20260811 17:24:12.361901 12715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:24:12.367759 12715 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.190" port: 45409 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.157.188@KRBTEST.COM; defaulting to no policy
I20260811 17:24:12.372577 12603 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.189" port: 37241 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.188@KRBTEST.COM" created.
Aug 11 17:24:12 dist-test-slave-mjjr krb5kdc[12472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469052, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for kudu/127.9.157.189@KRBTEST.COM
Aug 11 17:24:12 dist-test-slave-mjjr krb5kdc[12472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469052, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.189@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:24:12.385185 12605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.373224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:37241 (local address 127.0.0.1:47024)
0811 17:24:12.373427 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.373443 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:24:12.373517 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:24:12.374252 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:24:12.374258 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:24:12.374481 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:24:12.374652 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:12.374664 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.376069 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.376074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:12.376909 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:12.376919 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.377597 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.377601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:12.377706 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:12.378358 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:24:12.378378 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:24:12.380194 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:24:12.382927 (+ 2733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:12.382932 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:12.382934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:12.383201 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:12.383623 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:12.383626 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:12.383628 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:12.383749 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:12.384578 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:24:12.384581 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:24:12.384696 (+ 115us) client_negotiation.cc:769] Sending connection context
0811 17:24:12.384818 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 17:24:12.384992 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
I20260811 17:24:12.385648 12607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.370379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39710 (local address 127.9.157.190:45409)
0811 17:24:12.370544 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:12.370552 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:12.371467 (+ 915us) server_negotiation.cc:407] Connection header received
0811 17:24:12.372511 (+ 1044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:12.372514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:12.372584 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:12.372693 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:12.374312 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.374958 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.376773 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.377944 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.378072 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:12.381174 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:12.381180 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:12.381189 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:12.381213 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:12.382946 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.383546 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.383550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.383552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.383665 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.384061 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.384065 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.384067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.384294 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:12.384406 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:12.384859 (+ 453us) server_negotiation.cc:299] Negotiation successful
0811 17:24:12.385446 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":450,"server-negotiator.queue_time_us":63}
I20260811 17:24:12.386255 12723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.373405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47024 (local address 127.9.157.189:37241)
0811 17:24:12.373786 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:12.373797 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:12.373821 (+ 24us) server_negotiation.cc:407] Connection header received
0811 17:24:12.373862 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:12.373868 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:12.374054 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:12.374222 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:12.374828 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.375903 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.377048 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.377459 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.377649 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:12.380398 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:12.380408 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:12.380423 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:12.380458 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:12.382792 (+ 2334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.383342 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.383351 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.383356 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.383512 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.383865 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.383870 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.383872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.384449 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:12.384619 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:12.384912 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 17:24:12.385164 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":187,"threads_started":1}
I20260811 17:24:12.386250 12720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.370762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:45409 (local address 127.0.0.1:39710)
0811 17:24:12.371293 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.371343 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 17:24:12.372284 (+ 941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:24:12.372922 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:24:12.372929 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:24:12.373434 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:24:12.374143 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:12.374164 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.375094 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.375100 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:12.376628 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:12.376641 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.378123 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.378131 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:12.378342 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:12.378918 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:24:12.378935 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:24:12.381033 (+ 2098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:24:12.383088 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:12.383099 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:12.383112 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:12.383428 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:12.383784 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:12.383789 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:12.383791 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:12.383936 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:12.384427 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:24:12.384432 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:24:12.384570 (+ 138us) client_negotiation.cc:769] Sending connection context
0811 17:24:12.384762 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 17:24:12.385103 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":126,"thread_start_us":142,"threads_started":1}
I20260811 17:24:12.387854 12715 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.189" port: 37241 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:24:12.389150 12603 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:24:12.389961 12605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.389663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:45039 (local address 127.0.0.1:48738)
0811 17:24:12.389829 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.389893 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:45039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260811 17:24:12.390771 12605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.390442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:45039 (local address 127.0.0.1:48744)
0811 17:24:12.390637 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.390695 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:45039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260811 17:24:12.390962 12603 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.188:45039: Network error: Client connection negotiation failed: client connection to 127.9.157.188:45039: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.188.keytab.
Entry for principal kudu/127.9.157.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.188.keytab.
I20260811 17:24:12.393149 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.188:45039
--webserver_interface=127.9.157.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.188.keytab
--principal=kudu/127.9.157.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:44059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:45409,127.9.157.189:37241,127.9.157.188:45039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 17:24:12 dist-test-slave-mjjr krb5kdc[12472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469052, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.189@KRBTEST.COM for kudu/127.9.157.189@KRBTEST.COM
I20260811 17:24:12.398010 12720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.388401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:37241 (local address 127.0.0.1:47034)
0811 17:24:12.388556 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.388573 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:24:12.388663 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:24:12.388914 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:24:12.388917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:24:12.389122 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:24:12.389243 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:12.389249 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.390259 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.390262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:12.391121 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:12.391133 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.391678 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.391685 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:12.391792 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:12.392391 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:24:12.392403 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:24:12.393856 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:24:12.396206 (+ 2350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:12.396214 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:12.396218 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:12.396527 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:12.396807 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:12.396812 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:12.396815 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:12.396927 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:12.397320 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:24:12.397325 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:24:12.397459 (+ 134us) client_negotiation.cc:769] Sending connection context
0811 17:24:12.397595 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 17:24:12.397799 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 17:24:12.398209 12723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.388597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47034 (local address 127.9.157.189:37241)
0811 17:24:12.388712 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:12.388716 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:12.388727 (+ 11us) server_negotiation.cc:407] Connection header received
0811 17:24:12.388757 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:12.388759 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:12.388795 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:12.388863 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:12.389406 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.389981 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.391265 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.391510 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.391635 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:12.394005 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:12.394012 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:12.394018 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:12.394052 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:12.396084 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.396635 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.396637 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.396638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.396711 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.397033 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.397035 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.397036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.397221 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:12.397295 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:12.397686 (+ 391us) server_negotiation.cc:299] Negotiation successful
0811 17:24:12.397918 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":46}
I20260811 17:24:12.399123 12715 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:24:12.400038 12720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.399699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:45039 (local address 127.0.0.1:48748)
0811 17:24:12.399843 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.399942 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:45039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 17:24:12.400976 12641 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.157.188:45039: connect: Connection refused (error 111)
I20260811 17:24:12.402043 12720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.401804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:45039 (local address 127.0.0.1:48762)
0811 17:24:12.401938 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.401987 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:45039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 17:24:12.402311 12715 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.188:45039: Network error: Client connection negotiation failed: client connection to 127.9.157.188:45039: connect: Connection refused (error 111)
I20260811 17:24:12.422351 12603 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 } attempt: 1
I20260811 17:24:12.423346 12605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.423027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:45039 (local address 127.0.0.1:48766)
0811 17:24:12.423194 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.423260 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:45039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 17:24:12.424157 12605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.423928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:45039 (local address 127.0.0.1:48772)
0811 17:24:12.424044 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.424099 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:45039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 17:24:12.424296 12603 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.188:45039: Network error: Client connection negotiation failed: client connection to 127.9.157.188:45039: connect: Connection refused (error 111)
I20260811 17:24:12.451604 12715 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 } attempt: 1
I20260811 17:24:12.452661 12720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.452341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:45039 (local address 127.0.0.1:48788)
0811 17:24:12.452511 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.452576 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:45039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 17:24:12.453428 12720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.453216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:45039 (local address 127.0.0.1:48802)
0811 17:24:12.453324 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.453369 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:45039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 17:24:12.453584 12715 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.188:45039: Network error: Client connection negotiation failed: client connection to 127.9.157.188:45039: connect: Connection refused (error 111)
I20260811 17:24:12.499485 12603 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 } attempt: 2
I20260811 17:24:12.500443 12605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.500123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:45039 (local address 127.0.0.1:48806)
0811 17:24:12.500284 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.500356 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:45039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 17:24:12.501207 12605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.500976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:45039 (local address 127.0.0.1:48812)
0811 17:24:12.501095 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.501148 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:45039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 17:24:12.501379 12603 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.188:45039: Network error: Client connection negotiation failed: client connection to 127.9.157.188:45039: connect: Connection refused (error 111)
W20260811 17:24:12.517921 12725 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 17:24:12.518232 12725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:24:12.518256 12725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:24:12.518337 12725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 17:24:12.521793 12715 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 } attempt: 2
I20260811 17:24:12.522744 12720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.522420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:45039 (local address 127.0.0.1:48822)
0811 17:24:12.522590 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.522659 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:45039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260811 17:24:12.522836 12725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:24:12.522929 12725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:24:12.522966 12725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:24:12.523006 12725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:24:12.523034 12725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:24:12.523746 12720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.523476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:45039 (local address 127.0.0.1:48830)
0811 17:24:12.523585 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.523683 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:45039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 17:24:12.523988 12715 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.188:45039: Network error: Client connection negotiation failed: client connection to 127.9.157.188:45039: connect: Connection refused (error 111)
I20260811 17:24:12.528990 12725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:44059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:45409,127.9.157.189:37241,127.9.157.188:45039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.188.keytab
--principal=kudu/127.9.157.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.188:45039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.157.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:24:12.530210 12725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:24:12.531316 12725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:24:12.540323 12736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:24:12.540405 12739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:24:12.540558 12725 server_base.cc:1034] running on GCE node
W20260811 17:24:12.540329 12742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:24:12.541347 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:24:12.542646 12725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:24:12.543850 12725 hybrid_clock.cc:648] HybridClock initialized: now 1786469052543828 us; error 32 us; skew 500 ppm
Aug 11 17:24:12 dist-test-slave-mjjr krb5kdc[12472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469052, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:24:12.546993 12725 init.cc:377] Logged in from keytab as kudu/127.9.157.188@KRBTEST.COM (short username kudu)
I20260811 17:24:12.548084 12725 webserver.cc:460] Webserver started at http://127.9.157.188:43989/ using document root <none> and password file <none>
I20260811 17:24:12.548686 12725 fs_manager.cc:359] Metadata directory not provided
I20260811 17:24:12.548779 12725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:24:12.549028 12725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:24:12.551188 12725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-2/data/instance:
uuid: "da2d0cdfa68c47a4a2ed4ee49c591350"
format_stamp: "Formatted at 2026-08-11 17:24:12 on dist-test-slave-mjjr"
server_key: "f5b3748f7ea6026fe588b08d685f6ac9"
server_key_iv: "db2bca13d22f258eb03493af72ea56b5"
server_key_version: "encryptionkey@0"
I20260811 17:24:12.551842 12725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-2/wal/instance:
uuid: "da2d0cdfa68c47a4a2ed4ee49c591350"
format_stamp: "Formatted at 2026-08-11 17:24:12 on dist-test-slave-mjjr"
server_key: "f5b3748f7ea6026fe588b08d685f6ac9"
server_key_iv: "db2bca13d22f258eb03493af72ea56b5"
server_key_version: "encryptionkey@0"
I20260811 17:24:12.555893 12725 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:24:12.558705 12753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:24:12.560184 12725 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 17:24:12.560345 12725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-2/wal
uuid: "da2d0cdfa68c47a4a2ed4ee49c591350"
format_stamp: "Formatted at 2026-08-11 17:24:12 on dist-test-slave-mjjr"
server_key: "f5b3748f7ea6026fe588b08d685f6ac9"
server_key_iv: "db2bca13d22f258eb03493af72ea56b5"
server_key_version: "encryptionkey@0"
I20260811 17:24:12.560473 12725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:24:12.593971 12725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:24:12.597519 12725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:24:12.597769 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:24:12.600567 12603 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 } attempt: 3
I20260811 17:24:12.601464 12605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.601139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:45039 (local address 127.0.0.1:48836)
0811 17:24:12.601284 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.601359 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:45039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 17:24:12.602409 12605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.602097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:45039 (local address 127.0.0.1:48850)
0811 17:24:12.602267 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.602332 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:45039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260811 17:24:12.602591 12603 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.188:45039: Network error: Client connection negotiation failed: client connection to 127.9.157.188:45039: connect: Connection refused (error 111)
I20260811 17:24:12.606372 12725 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.188:45039
I20260811 17:24:12.606711 12838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.188:45039 every 8 connection(s)
I20260811 17:24:12.607527 12725 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-2/data/info.pb
I20260811 17:24:12.610796 12840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:24:12.614192 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 12725
I20260811 17:24:12.614355 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786468743878666-9846-0/minicluster-data/master-2/wal/instance
I20260811 17:24:12.614655 9846 external_mini_cluster.cc:1428] Setting key df995ea5548c2845cfa29aa7427540e3
I20260811 17:24:12.615198 12715 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 } attempt: 3
I20260811 17:24:12.615031 12840 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.190" port: 45409 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 17:24:12 dist-test-slave-mjjr krb5kdc[12472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469052, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.189@KRBTEST.COM for kudu/127.9.157.188@KRBTEST.COM
Aug 11 17:24:12 dist-test-slave-mjjr krb5kdc[12472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469052, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.188@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
Aug 11 17:24:12 dist-test-slave-mjjr krb5kdc[12472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469051, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.188@KRBTEST.COM
I20260811 17:24:12.630641 12720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.615676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:45039 (local address 127.0.0.1:48870)
0811 17:24:12.615880 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.615898 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:24:12.616000 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:24:12.618231 (+ 2231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:24:12.618234 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:24:12.618475 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:24:12.618619 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:12.618631 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.620660 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.620664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:12.621219 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:12.621229 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.622210 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.622219 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:12.622343 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:12.623354 (+ 1011us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:24:12.623416 (+ 62us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:24:12.625118 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:24:12.628296 (+ 3178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:12.628303 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:12.628308 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:12.628608 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:12.629104 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:12.629108 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:12.629110 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:12.629214 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:12.630079 (+ 865us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:24:12.630083 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:24:12.630194 (+ 111us) client_negotiation.cc:769] Sending connection context
0811 17:24:12.630280 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 17:24:12.630457 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
I20260811 17:24:12.631187 12607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.617123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39712 (local address 127.9.157.190:45409)
0811 17:24:12.617271 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:12.617276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:12.617535 (+ 259us) server_negotiation.cc:407] Connection header received
0811 17:24:12.617938 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:12.617941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:12.618006 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:12.618101 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:12.619143 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.619714 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.621669 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.622000 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.622134 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:12.626061 (+ 3927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:12.626067 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:12.626073 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:12.626114 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:12.628231 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.629014 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.629021 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.629026 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.629129 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.629960 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.629964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.629966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.630204 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:12.630306 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:12.630849 (+ 543us) server_negotiation.cc:299] Negotiation successful
0811 17:24:12.631015 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 17:24:12.632586 12844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.616140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48854 (local address 127.9.157.188:45039)
0811 17:24:12.616636 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:12.616657 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:12.616700 (+ 43us) server_negotiation.cc:407] Connection header received
0811 17:24:12.617694 (+ 994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:12.617722 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:12.618018 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:12.619118 (+ 1100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:12.619198 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.620438 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.621832 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.622170 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.622334 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:12.626602 (+ 4268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:12.626608 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:12.626613 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:12.626649 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:12.629424 (+ 2775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.629927 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.629929 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.629930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.630012 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.630371 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.630375 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.630376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.630576 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:12.630814 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:12.631085 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 17:24:12.631571 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":401,"server-negotiator.queue_time_us":336,"spinlock_wait_cycles":3328,"thread_start_us":159,"threads_started":1}
I20260811 17:24:12.632591 12847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.617035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:45409 (local address 127.0.0.1:39712)
0811 17:24:12.617378 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.617430 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 17:24:12.617729 (+ 299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:24:12.618155 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:24:12.618163 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:24:12.618519 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:24:12.619001 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:12.619011 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.619828 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.619832 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:12.621413 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:12.621439 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.622811 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.622816 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:12.623111 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:12.623785 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:24:12.623804 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:24:12.625867 (+ 2063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:24:12.628368 (+ 2501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:12.628382 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:12.628395 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:12.628862 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:12.629251 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:12.629255 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:12.629258 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:12.629424 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:12.630379 (+ 955us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:24:12.630387 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:24:12.630551 (+ 164us) client_negotiation.cc:769] Sending connection context
0811 17:24:12.631498 (+ 947us) client_negotiation.cc:240] Negotiation successful
0811 17:24:12.631706 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":160,"thread_start_us":96,"threads_started":1}
I20260811 17:24:12.632591 12845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.616278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48870 (local address 127.9.157.188:45039)
0811 17:24:12.616636 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:12.616657 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:12.616700 (+ 43us) server_negotiation.cc:407] Connection header received
0811 17:24:12.617694 (+ 994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:12.617722 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:12.618018 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:12.618807 (+ 789us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:12.618946 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.620536 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.621406 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.622010 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.622190 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:12.625314 (+ 3124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:12.625324 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:12.625344 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:12.625379 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:12.628129 (+ 2750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.628776 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.628783 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.628785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.628980 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.629335 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.629337 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.629339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.629951 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:12.631164 (+ 1213us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:12.631248 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 17:24:12.631463 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":201,"server-negotiator.queue_time_us":302,"spinlock_wait_cycles":10240,"thread_start_us":172,"threads_started":1}
I20260811 17:24:12.634609 12840 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.189" port: 37241 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:24:12.637022 12715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 935ef3abfeee42fe83b0fccc2d4dad96: Bootstrap starting.
I20260811 17:24:12.639786 12715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 935ef3abfeee42fe83b0fccc2d4dad96: Neither blocks nor log segments found. Creating new log.
Aug 11 17:24:12 dist-test-slave-mjjr krb5kdc[12472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469051, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:24:12.640661 12715 log.cc:824] T 00000000000000000000000000000000 P 935ef3abfeee42fe83b0fccc2d4dad96: Log is configured to *not* fsync() on all Append() calls
I20260811 17:24:12.643146 12715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 935ef3abfeee42fe83b0fccc2d4dad96: No bootstrap required, opened a new log
I20260811 17:24:12.644520 12607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.635243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39726 (local address 127.9.157.190:45409)
0811 17:24:12.635382 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:12.635387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:12.635500 (+ 113us) server_negotiation.cc:407] Connection header received
0811 17:24:12.635619 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:12.635621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:12.635701 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:12.635804 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:12.636472 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.636997 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.637803 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.638046 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.638161 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:12.640741 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:12.640746 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:12.640749 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:12.640801 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:12.642496 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.642966 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.642970 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.642972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.643133 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.643436 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.643439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.643441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.643740 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:12.643984 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:12.644231 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 17:24:12.644365 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Aug 11 17:24:12 dist-test-slave-mjjr krb5kdc[12472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469052, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.188@KRBTEST.COM for kudu/127.9.157.189@KRBTEST.COM
I20260811 17:24:12.648001 12715 raft_consensus.cc:348] T 00000000000000000000000000000000 P 935ef3abfeee42fe83b0fccc2d4dad96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "682ca2251d7449118763f40113fd3fd4" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 45409 } } peers { permanent_uuid: "935ef3abfeee42fe83b0fccc2d4dad96" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 37241 } } peers { permanent_uuid: "da2d0cdfa68c47a4a2ed4ee49c591350" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 } }
I20260811 17:24:12.648281 12715 raft_consensus.cc:729] T 00000000000000000000000000000000 P 935ef3abfeee42fe83b0fccc2d4dad96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 935ef3abfeee42fe83b0fccc2d4dad96, State: Initialized, Role: FOLLOWER
I20260811 17:24:12.648624 12715 consensus_queue.cc:260] T 00000000000000000000000000000000 P 935ef3abfeee42fe83b0fccc2d4dad96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "682ca2251d7449118763f40113fd3fd4" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 45409 } } peers { permanent_uuid: "935ef3abfeee42fe83b0fccc2d4dad96" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 37241 } } peers { permanent_uuid: "da2d0cdfa68c47a4a2ed4ee49c591350" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 } }
I20260811 17:24:12.649118 12850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 935ef3abfeee42fe83b0fccc2d4dad96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "682ca2251d7449118763f40113fd3fd4" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 45409 } } peers { permanent_uuid: "935ef3abfeee42fe83b0fccc2d4dad96" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 37241 } } peers { permanent_uuid: "da2d0cdfa68c47a4a2ed4ee49c591350" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 } } }
I20260811 17:24:12.649335 12850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 935ef3abfeee42fe83b0fccc2d4dad96 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:24:12.650190 12715 sys_catalog.cc:564] T 00000000000000000000000000000000 P 935ef3abfeee42fe83b0fccc2d4dad96 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:24:12.651051 12723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.635430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47040 (local address 127.9.157.189:37241)
0811 17:24:12.635542 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:12.635546 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:12.641334 (+ 5788us) server_negotiation.cc:407] Connection header received
0811 17:24:12.641522 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:12.641525 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:12.641576 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:12.641676 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:12.642294 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.642992 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.644075 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.644434 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.644569 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:12.647324 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:12.647328 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:12.647331 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:12.647357 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:12.649166 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.649654 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.649658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.649660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.649749 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.650073 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.650076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.650078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.650363 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:12.650467 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:12.650782 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 17:24:12.650908 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 17:24:12.651052 12847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.635190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:37241 (local address 127.0.0.1:47040)
0811 17:24:12.641241 (+ 6051us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.641260 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 17:24:12.641412 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:24:12.641834 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:24:12.641837 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:24:12.642033 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:24:12.642153 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:12.642161 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.643136 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.643139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:12.643960 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:12.643971 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.644573 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.644577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:12.644693 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:12.645212 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:24:12.645227 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:24:12.647143 (+ 1916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:24:12.649292 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:12.649295 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:12.649297 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:12.649548 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:12.649861 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:12.649864 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:12.649866 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:12.649972 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:12.650463 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:24:12.650467 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:24:12.650589 (+ 122us) client_negotiation.cc:769] Sending connection context
0811 17:24:12.650724 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 17:24:12.650874 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5942}
I20260811 17:24:12.652119 12840 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 17:24:12 dist-test-slave-mjjr krb5kdc[12472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469051, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.189@KRBTEST.COM
W20260811 17:24:12.656404 12867 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 935ef3abfeee42fe83b0fccc2d4dad96: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:24:12.656596 12867 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 11 17:24:12 dist-test-slave-mjjr krb5kdc[12472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469052, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.188@KRBTEST.COM for kudu/127.9.157.188@KRBTEST.COM
I20260811 17:24:12.657989 12849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.646495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47052 (local address 127.9.157.189:37241)
0811 17:24:12.646898 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:12.646908 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:12.646921 (+ 13us) server_negotiation.cc:407] Connection header received
0811 17:24:12.646960 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:12.646962 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:12.647002 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:12.647106 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:12.647876 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.648641 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.649785 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.650165 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.650276 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:12.653273 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:12.653277 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:12.653280 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:12.653321 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:12.655308 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.655877 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.655881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.655883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.655991 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.656327 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.656331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.656333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.656614 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:12.656872 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:12.657661 (+ 789us) server_negotiation.cc:299] Negotiation successful
0811 17:24:12.657791 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":147,"threads_started":1}
I20260811 17:24:12.654366 12613 master_runner.cc:186] Error getting master registration for 127.9.157.190:45409: OK, instance_id { permanent_uuid: "682ca2251d7449118763f40113fd3fd4" instance_seqno: 1786469052126930 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 17:24:12.658442 12613 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 17:24:12.661794 9846 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:24:12.661880 12844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.652845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48884 (local address 127.9.157.188:45039)
0811 17:24:12.652964 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:12.652969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:12.652983 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:24:12.653018 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:12.653023 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:12.653071 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:12.653147 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:12.653674 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.654546 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.655631 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.656006 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.656100 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:12.658444 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:12.658450 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:12.658454 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:12.658478 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:12.660109 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.660590 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.660596 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.660598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.660680 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.660957 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.660960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.660962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.661226 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:12.661330 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:12.661579 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 17:24:12.661724 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":41}
I20260811 17:24:12.661880 12847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.652666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:45039 (local address 127.0.0.1:48884)
0811 17:24:12.652769 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.652784 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 17:24:12.652868 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:24:12.653262 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:24:12.653265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:24:12.653437 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:24:12.653546 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:12.653556 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.654676 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.654681 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:12.655510 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:12.655522 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.656210 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.656213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:12.656304 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:12.656761 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:24:12.656780 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:24:12.658301 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:24:12.660219 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:12.660222 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:12.660224 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:12.660482 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:12.660773 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:12.660777 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:12.660779 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:12.660864 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:12.661321 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:24:12.661324 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:24:12.661418 (+ 94us) client_negotiation.cc:769] Sending connection context
0811 17:24:12.661523 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 17:24:12.661668 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 17:24:12.663487 12561 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "682ca2251d7449118763f40113fd3fd4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39726
I20260811 17:24:12.664968 12840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da2d0cdfa68c47a4a2ed4ee49c591350: Bootstrap starting.
I20260811 17:24:12.667308 12840 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da2d0cdfa68c47a4a2ed4ee49c591350: Neither blocks nor log segments found. Creating new log.
I20260811 17:24:12.668134 12840 log.cc:824] T 00000000000000000000000000000000 P da2d0cdfa68c47a4a2ed4ee49c591350: Log is configured to *not* fsync() on all Append() calls
I20260811 17:24:12.670759 12840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da2d0cdfa68c47a4a2ed4ee49c591350: No bootstrap required, opened a new log
I20260811 17:24:12.673921 12840 raft_consensus.cc:348] T 00000000000000000000000000000000 P da2d0cdfa68c47a4a2ed4ee49c591350 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "682ca2251d7449118763f40113fd3fd4" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 45409 } } peers { permanent_uuid: "935ef3abfeee42fe83b0fccc2d4dad96" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 37241 } } peers { permanent_uuid: "da2d0cdfa68c47a4a2ed4ee49c591350" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 } }
I20260811 17:24:12.674166 12840 raft_consensus.cc:729] T 00000000000000000000000000000000 P da2d0cdfa68c47a4a2ed4ee49c591350 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da2d0cdfa68c47a4a2ed4ee49c591350, State: Initialized, Role: FOLLOWER
I20260811 17:24:12.674590 12840 consensus_queue.cc:260] T 00000000000000000000000000000000 P da2d0cdfa68c47a4a2ed4ee49c591350 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "682ca2251d7449118763f40113fd3fd4" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 45409 } } peers { permanent_uuid: "935ef3abfeee42fe83b0fccc2d4dad96" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 37241 } } peers { permanent_uuid: "da2d0cdfa68c47a4a2ed4ee49c591350" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 } }
I20260811 17:24:12.675196 12871 sys_catalog.cc:455] T 00000000000000000000000000000000 P da2d0cdfa68c47a4a2ed4ee49c591350 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "682ca2251d7449118763f40113fd3fd4" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 45409 } } peers { permanent_uuid: "935ef3abfeee42fe83b0fccc2d4dad96" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 37241 } } peers { permanent_uuid: "da2d0cdfa68c47a4a2ed4ee49c591350" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 } } }
I20260811 17:24:12.675413 12871 sys_catalog.cc:458] T 00000000000000000000000000000000 P da2d0cdfa68c47a4a2ed4ee49c591350 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:24:12.676134 12840 sys_catalog.cc:564] T 00000000000000000000000000000000 P da2d0cdfa68c47a4a2ed4ee49c591350 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 17:24:12.682150 12894 catalog_manager.cc:1558] T 00000000000000000000000000000000 P da2d0cdfa68c47a4a2ed4ee49c591350: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:24:12.682328 12894 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:24:12.680614 12725 master_runner.cc:186] Error getting master registration for 127.9.157.190:45409: OK, instance_id { permanent_uuid: "682ca2251d7449118763f40113fd3fd4" instance_seqno: 1786469052126930 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 17:24:12.683625 12725 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 17:24:12.766775 12603 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 } attempt: 4
Aug 11 17:24:12 dist-test-slave-mjjr krb5kdc[12472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469052, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for kudu/127.9.157.188@KRBTEST.COM
I20260811 17:24:12.776439 12605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.767429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:45039 (local address 127.0.0.1:48886)
0811 17:24:12.767646 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:12.767685 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 17:24:12.767835 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:24:12.768134 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:24:12.768136 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:24:12.768358 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:24:12.768458 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:12.768463 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.769474 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.769477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:12.770017 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:12.770026 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.770659 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.770663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:12.770753 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:12.771308 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:24:12.771321 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:24:12.772908 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:24:12.774709 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:12.774714 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:12.774717 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:12.774989 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:12.775269 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:12.775273 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:12.775277 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:12.775383 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:12.775853 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:24:12.775856 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:24:12.775975 (+ 119us) client_negotiation.cc:769] Sending connection context
0811 17:24:12.776083 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 17:24:12.776246 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144}
I20260811 17:24:12.776446 12844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.767490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48886 (local address 127.9.157.188:45039)
0811 17:24:12.767622 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:12.767627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:12.767772 (+ 145us) server_negotiation.cc:407] Connection header received
0811 17:24:12.767982 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:12.767985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:12.768040 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:12.768112 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:12.768586 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.769370 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.770146 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.770550 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.770649 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:12.773056 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:12.773059 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:12.773062 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:12.773086 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:12.774579 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.775081 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.775085 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.775088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.775168 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.775478 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.775481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.775483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.775742 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:12.775863 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:12.776163 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 17:24:12.776314 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 17:24:12.780076 12603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 682ca2251d7449118763f40113fd3fd4: Bootstrap starting.
I20260811 17:24:12.783303 12603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 682ca2251d7449118763f40113fd3fd4: Neither blocks nor log segments found. Creating new log.
I20260811 17:24:12.784292 12603 log.cc:824] T 00000000000000000000000000000000 P 682ca2251d7449118763f40113fd3fd4: Log is configured to *not* fsync() on all Append() calls
I20260811 17:24:12.786620 12603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 682ca2251d7449118763f40113fd3fd4: No bootstrap required, opened a new log
I20260811 17:24:12.789889 12603 raft_consensus.cc:348] T 00000000000000000000000000000000 P 682ca2251d7449118763f40113fd3fd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "682ca2251d7449118763f40113fd3fd4" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 45409 } } peers { permanent_uuid: "935ef3abfeee42fe83b0fccc2d4dad96" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 37241 } } peers { permanent_uuid: "da2d0cdfa68c47a4a2ed4ee49c591350" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 } }
I20260811 17:24:12.790130 12603 raft_consensus.cc:729] T 00000000000000000000000000000000 P 682ca2251d7449118763f40113fd3fd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 682ca2251d7449118763f40113fd3fd4, State: Initialized, Role: FOLLOWER
I20260811 17:24:12.790475 12603 consensus_queue.cc:260] T 00000000000000000000000000000000 P 682ca2251d7449118763f40113fd3fd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "682ca2251d7449118763f40113fd3fd4" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 45409 } } peers { permanent_uuid: "935ef3abfeee42fe83b0fccc2d4dad96" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 37241 } } peers { permanent_uuid: "da2d0cdfa68c47a4a2ed4ee49c591350" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 } }
I20260811 17:24:12.791071 12903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 682ca2251d7449118763f40113fd3fd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "682ca2251d7449118763f40113fd3fd4" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 45409 } } peers { permanent_uuid: "935ef3abfeee42fe83b0fccc2d4dad96" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 37241 } } peers { permanent_uuid: "da2d0cdfa68c47a4a2ed4ee49c591350" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 } } }
I20260811 17:24:12.791332 12903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 682ca2251d7449118763f40113fd3fd4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:24:12.791975 12603 sys_catalog.cc:564] T 00000000000000000000000000000000 P 682ca2251d7449118763f40113fd3fd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:24:12.792042 12561 raft_consensus.cc:482] T 00000000000000000000000000000000 P 682ca2251d7449118763f40113fd3fd4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 17:24:12.792138 12561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 682ca2251d7449118763f40113fd3fd4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:24:12.793594 12561 raft_consensus.cc:504] T 00000000000000000000000000000000 P 682ca2251d7449118763f40113fd3fd4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "682ca2251d7449118763f40113fd3fd4" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 45409 } } peers { permanent_uuid: "935ef3abfeee42fe83b0fccc2d4dad96" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 37241 } } peers { permanent_uuid: "da2d0cdfa68c47a4a2ed4ee49c591350" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 } }
I20260811 17:24:12.794677 12561 leader_election.cc:288] T 00000000000000000000000000000000 P 682ca2251d7449118763f40113fd3fd4 [CANDIDATE]: Term 1 election: Requested vote from peers 935ef3abfeee42fe83b0fccc2d4dad96 (127.9.157.189:37241), da2d0cdfa68c47a4a2ed4ee49c591350 (127.9.157.188:45039)
I20260811 17:24:12.795457 12676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "682ca2251d7449118763f40113fd3fd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "935ef3abfeee42fe83b0fccc2d4dad96"
I20260811 17:24:12.795913 12676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 935ef3abfeee42fe83b0fccc2d4dad96 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:24:12.796821 12796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "682ca2251d7449118763f40113fd3fd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "da2d0cdfa68c47a4a2ed4ee49c591350"
I20260811 17:24:12.797062 12796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da2d0cdfa68c47a4a2ed4ee49c591350 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:24:12.797519 12676 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 935ef3abfeee42fe83b0fccc2d4dad96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 682ca2251d7449118763f40113fd3fd4 in term 1.
I20260811 17:24:12.798419 12796 raft_consensus.cc:2454] T 00000000000000000000000000000000 P da2d0cdfa68c47a4a2ed4ee49c591350 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 682ca2251d7449118763f40113fd3fd4 in term 1.
I20260811 17:24:12.799014 12519 leader_election.cc:302] T 00000000000000000000000000000000 P 682ca2251d7449118763f40113fd3fd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 682ca2251d7449118763f40113fd3fd4, da2d0cdfa68c47a4a2ed4ee49c591350; no voters:
W20260811 17:24:12.802330 12921 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 682ca2251d7449118763f40113fd3fd4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:24:12.802547 12921 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:24:12.805486 12903 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 682ca2251d7449118763f40113fd3fd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:24:12.805796 12903 raft_consensus.cc:686] T 00000000000000000000000000000000 P 682ca2251d7449118763f40113fd3fd4 [term 1 LEADER]: Becoming Leader. State: Replica: 682ca2251d7449118763f40113fd3fd4, State: Running, Role: LEADER
I20260811 17:24:12.809026 12495 master_runner.cc:263] No leader master found from master 682ca2251d7449118763f40113fd3fd4
I20260811 17:24:12.809118 12495 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 17:24:12.809567 12903 consensus_queue.cc:237] T 00000000000000000000000000000000 P 682ca2251d7449118763f40113fd3fd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "682ca2251d7449118763f40113fd3fd4" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 45409 } } peers { permanent_uuid: "935ef3abfeee42fe83b0fccc2d4dad96" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 37241 } } peers { permanent_uuid: "da2d0cdfa68c47a4a2ed4ee49c591350" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 } }
I20260811 17:24:12.811813 12903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 682ca2251d7449118763f40113fd3fd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 682ca2251d7449118763f40113fd3fd4. Latest consensus state: current_term: 1 leader_uuid: "682ca2251d7449118763f40113fd3fd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "682ca2251d7449118763f40113fd3fd4" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 45409 } } peers { permanent_uuid: "935ef3abfeee42fe83b0fccc2d4dad96" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 37241 } } peers { permanent_uuid: "da2d0cdfa68c47a4a2ed4ee49c591350" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 } } }
I20260811 17:24:12.813112 12903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 682ca2251d7449118763f40113fd3fd4 [sys.catalog]: This master's current role is: LEADER
I20260811 17:24:12.824018 12943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:24:12.826071 12943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:24:12.830349 12844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.813773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48890 (local address 127.9.157.188:45039)
0811 17:24:12.813902 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:12.813907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:12.816936 (+ 3029us) server_negotiation.cc:407] Connection header received
0811 17:24:12.817125 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:12.817129 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:12.817182 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:12.817235 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:12.820327 (+ 3092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.821001 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.823011 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.823481 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.823579 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:12.825571 (+ 1992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:12.825576 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:12.825579 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:12.825612 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:12.827876 (+ 2264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.828550 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.828556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.828559 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.828675 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.829084 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.829091 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.829093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.829393 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:12.829540 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:12.829987 (+ 447us) server_negotiation.cc:299] Negotiation successful
0811 17:24:12.830134 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 17:24:12.834517 12849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.812390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47066 (local address 127.9.157.189:37241)
0811 17:24:12.812529 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:12.812534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:12.815217 (+ 2683us) server_negotiation.cc:407] Connection header received
0811 17:24:12.815475 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:12.815480 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:12.815539 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:12.815597 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:12.818262 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.819129 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.822109 (+ 2980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.822521 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.822629 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:12.827464 (+ 4835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:12.827472 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:12.827478 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:12.827522 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:12.829605 (+ 2083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.831406 (+ 1801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.831411 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.831412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.831547 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.833157 (+ 1610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.833161 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.833162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.833459 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:12.833637 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:12.834103 (+ 466us) server_negotiation.cc:299] Negotiation successful
0811 17:24:12.834281 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 17:24:12.838940 12607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:12.814565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39728 (local address 127.9.157.190:45409)
0811 17:24:12.814678 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:12.814683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:12.821730 (+ 7047us) server_negotiation.cc:407] Connection header received
0811 17:24:12.821949 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:12.821953 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:12.822016 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:12.822081 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:12.825167 (+ 3086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.825938 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.830200 (+ 4262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:12.830560 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:12.830668 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:12.833159 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:12.833164 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:12.833167 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:12.833212 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:12.835082 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.837229 (+ 2147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.837233 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.837235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.837335 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:12.837663 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:12.837667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:12.837669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:12.837909 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:12.838034 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:12.838638 (+ 604us) server_negotiation.cc:299] Negotiation successful
0811 17:24:12.838772 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 17:24:12.843513 12796 raft_consensus.cc:1261] T 00000000000000000000000000000000 P da2d0cdfa68c47a4a2ed4ee49c591350 [term 1 FOLLOWER]: Refusing update from remote peer 682ca2251d7449118763f40113fd3fd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:24:12.844630 12676 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 935ef3abfeee42fe83b0fccc2d4dad96 [term 1 FOLLOWER]: Refusing update from remote peer 682ca2251d7449118763f40113fd3fd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:24:12.845449 12903 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 682ca2251d7449118763f40113fd3fd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "935ef3abfeee42fe83b0fccc2d4dad96" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 37241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:24:12.848639 12903 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 682ca2251d7449118763f40113fd3fd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da2d0cdfa68c47a4a2ed4ee49c591350" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 17:24:12.849484 12766 tablet.cc:2369] T 00000000000000000000000000000000 P da2d0cdfa68c47a4a2ed4ee49c591350: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:24:12.851608 12647 tablet.cc:2369] T 00000000000000000000000000000000 P 935ef3abfeee42fe83b0fccc2d4dad96: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:24:12.856478 12871 sys_catalog.cc:455] T 00000000000000000000000000000000 P da2d0cdfa68c47a4a2ed4ee49c591350 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 682ca2251d7449118763f40113fd3fd4. Latest consensus state: current_term: 1 leader_uuid: "682ca2251d7449118763f40113fd3fd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "682ca2251d7449118763f40113fd3fd4" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 45409 } } peers { permanent_uuid: "935ef3abfeee42fe83b0fccc2d4dad96" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 37241 } } peers { permanent_uuid: "da2d0cdfa68c47a4a2ed4ee49c591350" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 } } }
I20260811 17:24:12.856634 12871 sys_catalog.cc:458] T 00000000000000000000000000000000 P da2d0cdfa68c47a4a2ed4ee49c591350 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:24:12.861290 12850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 935ef3abfeee42fe83b0fccc2d4dad96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 682ca2251d7449118763f40113fd3fd4. Latest consensus state: current_term: 1 leader_uuid: "682ca2251d7449118763f40113fd3fd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "682ca2251d7449118763f40113fd3fd4" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 45409 } } peers { permanent_uuid: "935ef3abfeee42fe83b0fccc2d4dad96" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 37241 } } peers { permanent_uuid: "da2d0cdfa68c47a4a2ed4ee49c591350" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 } } }
I20260811 17:24:12.861442 12850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 935ef3abfeee42fe83b0fccc2d4dad96 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:24:12.871805 12934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 682ca2251d7449118763f40113fd3fd4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "682ca2251d7449118763f40113fd3fd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "682ca2251d7449118763f40113fd3fd4" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 45409 } } peers { permanent_uuid: "935ef3abfeee42fe83b0fccc2d4dad96" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 37241 } } peers { permanent_uuid: "da2d0cdfa68c47a4a2ed4ee49c591350" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 } } }
I20260811 17:24:12.871999 12934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 682ca2251d7449118763f40113fd3fd4 [sys.catalog]: This master's current role is: LEADER
I20260811 17:24:12.872380 12946 mvcc.cc:205] Tried to move back new op lower bound from 7317377240439009280 to 7317377240312586240. Current Snapshot: MvccSnapshot[applied={T|T < 7317377240439009280}]
I20260811 17:24:12.872402 12850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 935ef3abfeee42fe83b0fccc2d4dad96 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "682ca2251d7449118763f40113fd3fd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "682ca2251d7449118763f40113fd3fd4" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 45409 } } peers { permanent_uuid: "935ef3abfeee42fe83b0fccc2d4dad96" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 37241 } } peers { permanent_uuid: "da2d0cdfa68c47a4a2ed4ee49c591350" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 } } }
I20260811 17:24:12.872517 12850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 935ef3abfeee42fe83b0fccc2d4dad96 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:24:12.873253 12903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 682ca2251d7449118763f40113fd3fd4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "682ca2251d7449118763f40113fd3fd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "682ca2251d7449118763f40113fd3fd4" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 45409 } } peers { permanent_uuid: "935ef3abfeee42fe83b0fccc2d4dad96" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 37241 } } peers { permanent_uuid: "da2d0cdfa68c47a4a2ed4ee49c591350" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 } } }
I20260811 17:24:12.873373 12903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 682ca2251d7449118763f40113fd3fd4 [sys.catalog]: This master's current role is: LEADER
I20260811 17:24:12.874390 12871 sys_catalog.cc:455] T 00000000000000000000000000000000 P da2d0cdfa68c47a4a2ed4ee49c591350 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "682ca2251d7449118763f40113fd3fd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "682ca2251d7449118763f40113fd3fd4" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 45409 } } peers { permanent_uuid: "935ef3abfeee42fe83b0fccc2d4dad96" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 37241 } } peers { permanent_uuid: "da2d0cdfa68c47a4a2ed4ee49c591350" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 45039 } } }
I20260811 17:24:12.874511 12871 sys_catalog.cc:458] T 00000000000000000000000000000000 P da2d0cdfa68c47a4a2ed4ee49c591350 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:24:12.875725 12950 mvcc.cc:205] Tried to move back new op lower bound from 7317377240439009280 to 7317377240312586240. Current Snapshot: MvccSnapshot[applied={T|T < 7317377240439009280}]
I20260811 17:24:12.879518 12947 mvcc.cc:205] Tried to move back new op lower bound from 7317377240439009280 to 7317377240312586240. Current Snapshot: MvccSnapshot[applied={T|T < 7317377240439009280}]
I20260811 17:24:12.884196 12943 catalog_manager.cc:1347] Generated new cluster ID: fe29eb44ce824189b58c3b28bf58f752
I20260811 17:24:12.884300 12943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:24:12.904464 12943 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:24:12.905870 12943 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:24:12.914743 12943 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 682ca2251d7449118763f40113fd3fd4: Generated new TSK 0
I20260811 17:24:12.915632 12943 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 17:24:12.945247 12607 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:24:12.943839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39744 (local address 127.9.157.190:45409)
0811 17:24:12.943994 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:12.943999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:12.944017 (+ 18us) server_negotiation.cc:407] Connection header received
0811 17:24:12.944230 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:12.944234 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:12.944293 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:12.944362 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:12.945093 (+ 731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39744: BlockingRecv error: recv got EOF from 127.0.0.1:39744 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 17:24:12.945365 12844 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:24:12.943507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48896 (local address 127.9.157.188:45039)
0811 17:24:12.943675 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:12.943683 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:12.944387 (+ 704us) server_negotiation.cc:407] Connection header received
0811 17:24:12.944571 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:12.944574 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:12.944631 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:12.944689 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:12.945223 (+ 534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48896: BlockingRecv error: recv got EOF from 127.0.0.1:48896 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 17:24:12.945540 12849 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:24:12.943515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47068 (local address 127.9.157.189:37241)
0811 17:24:12.943687 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:12.943692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:12.944535 (+ 843us) server_negotiation.cc:407] Connection header received
0811 17:24:12.944729 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:12.944732 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:12.944781 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:12.944830 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:12.945414 (+ 584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47068: BlockingRecv error: recv got EOF from 127.0.0.1:47068 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 17:24:12.949074 12849 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:24:12.947952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47076 (local address 127.9.157.189:37241)
0811 17:24:12.948071 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:12.948074 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:12.948153 (+ 79us) server_negotiation.cc:407] Connection header received
0811 17:24:12.948335 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:12.948338 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:12.948402 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:12.948463 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:12.948931 (+ 468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47076: BlockingRecv error: recv got EOF from 127.0.0.1:47076 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 17:24:12.957625 12849 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:24:12.956558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47086 (local address 127.9.157.189:37241)
0811 17:24:12.956715 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:12.956720 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:12.956736 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:24:12.956796 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:12.956800 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:12.956855 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:12.956977 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:12.957461 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47086: BlockingRecv error: recv got EOF from 127.0.0.1:47086 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 17:24:12.962743 12849 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:24:12.961588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47088 (local address 127.9.157.189:37241)
0811 17:24:12.961716 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:12.961720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:12.961912 (+ 192us) server_negotiation.cc:407] Connection header received
0811 17:24:12.962059 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:12.962061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:12.962104 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:12.962172 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:12.962628 (+ 456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47088: BlockingRecv error: recv got EOF from 127.0.0.1:47088 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 17:24:12.970363 12849 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:24:12.969165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47102 (local address 127.9.157.189:37241)
0811 17:24:12.969395 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:12.969400 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:12.969514 (+ 114us) server_negotiation.cc:407] Connection header received
0811 17:24:12.969632 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:12.969636 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:12.969695 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:12.969799 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:12.970239 (+ 440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47102: BlockingRecv error: recv got EOF from 127.0.0.1:47102 (error 108)
Metrics: {"server-negotiator.queue_time_us":133}
W20260811 17:24:12.982101 12849 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:24:12.980921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47116 (local address 127.9.157.189:37241)
0811 17:24:12.981090 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:12.981095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:12.981219 (+ 124us) server_negotiation.cc:407] Connection header received
0811 17:24:12.981328 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:12.981332 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:12.981397 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:12.981507 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:12.981973 (+ 466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47116: BlockingRecv error: recv got EOF from 127.0.0.1:47116 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 17:24:13.001653 12849 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:24:13.000428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47128 (local address 127.9.157.189:37241)
0811 17:24:13.000587 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:13.000591 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:13.000734 (+ 143us) server_negotiation.cc:407] Connection header received
0811 17:24:13.000885 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:13.000888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:13.000938 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:13.001013 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:13.001523 (+ 510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47128: BlockingRecv error: recv got EOF from 127.0.0.1:47128 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 17:24:13.037212 12849 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:24:13.036103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47136 (local address 127.9.157.189:37241)
0811 17:24:13.036267 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:13.036271 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:13.036380 (+ 109us) server_negotiation.cc:407] Connection header received
0811 17:24:13.036512 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:13.036515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:13.036568 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:13.036654 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:13.037106 (+ 452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47136: BlockingRecv error: recv got EOF from 127.0.0.1:47136 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 17:24:13.104969 12849 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:24:13.103663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47142 (local address 127.9.157.189:37241)
0811 17:24:13.103872 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:13.103877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:13.104024 (+ 147us) server_negotiation.cc:407] Connection header received
0811 17:24:13.104207 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:13.104211 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:13.104270 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:13.104362 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:13.104852 (+ 490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47142: BlockingRecv error: recv got EOF from 127.0.0.1:47142 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
W20260811 17:24:13.236600 12849 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:24:13.235346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47152 (local address 127.9.157.189:37241)
0811 17:24:13.235563 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:13.235567 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:13.235655 (+ 88us) server_negotiation.cc:407] Connection header received
0811 17:24:13.235802 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:13.235804 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:13.235854 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:13.235932 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:13.236430 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47152: BlockingRecv error: recv got EOF from 127.0.0.1:47152 (error 108)
Metrics: {"server-negotiator.queue_time_us":114}
W20260811 17:24:13.496104 12849 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:24:13.495146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47168 (local address 127.9.157.189:37241)
0811 17:24:13.495290 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:13.495294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:13.495308 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:24:13.495364 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:13.495367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:13.495415 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:13.495499 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:13.495997 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47168: BlockingRecv error: recv got EOF from 127.0.0.1:47168 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 17:24:13.658154 12867 catalog_manager.cc:1259] Loaded cluster ID: fe29eb44ce824189b58c3b28bf58f752
I20260811 17:24:13.658252 12867 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 935ef3abfeee42fe83b0fccc2d4dad96: loading cluster ID for follower catalog manager: success
I20260811 17:24:13.660023 12867 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 935ef3abfeee42fe83b0fccc2d4dad96: acquiring CA information for follower catalog manager: success
I20260811 17:24:13.661175 12867 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 935ef3abfeee42fe83b0fccc2d4dad96: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 17:24:13.683884 12894 catalog_manager.cc:1259] Loaded cluster ID: fe29eb44ce824189b58c3b28bf58f752
I20260811 17:24:13.683979 12894 catalog_manager.cc:1552] T 00000000000000000000000000000000 P da2d0cdfa68c47a4a2ed4ee49c591350: loading cluster ID for follower catalog manager: success
I20260811 17:24:13.685817 12894 catalog_manager.cc:1574] T 00000000000000000000000000000000 P da2d0cdfa68c47a4a2ed4ee49c591350: acquiring CA information for follower catalog manager: success
I20260811 17:24:13.687122 12894 catalog_manager.cc:1602] T 00000000000000000000000000000000 P da2d0cdfa68c47a4a2ed4ee49c591350: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 17:24:14.014366 13066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:14.010597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47174 (local address 127.9.157.189:37241)
0811 17:24:14.010903 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:14.010907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:14.010923 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:24:14.010979 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:14.010981 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:14.011110 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:14.011212 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:24:14.011453 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:14.012019 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:14.012903 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:14.013130 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:14.013222 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:14.013522 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:24:14.014003 (+ 481us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:24:14.014101 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:14.014175 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 17:24:14.014235 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":97,"threads_started":1}
I20260811 17:24:14.021360 13074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:14.017823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48902 (local address 127.9.157.188:45039)
0811 17:24:14.018061 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:14.018065 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:14.018079 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:24:14.018180 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:14.018182 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:14.018304 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:14.018420 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:24:14.018603 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:14.019106 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:14.019981 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:14.020195 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:14.020284 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:14.020554 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:24:14.021019 (+ 465us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:24:14.021114 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:14.021175 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 17:24:14.021224 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":88,"threads_started":1}
I20260811 17:24:14.029414 13066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:14.024831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47186 (local address 127.9.157.189:37241)
0811 17:24:14.024987 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:14.024992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:14.025504 (+ 512us) server_negotiation.cc:407] Connection header received
0811 17:24:14.025628 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:14.025631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:14.025701 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:14.025780 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:24:14.026136 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:14.026700 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:14.028196 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:14.028531 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:14.028640 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:14.028887 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:24:14.028970 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:24:14.029054 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:14.029175 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 17:24:14.029237 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 17:24:14.029954 13085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:14.024831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39760 (local address 127.9.157.190:45409)
0811 17:24:14.025448 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:14.025452 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:14.025481 (+ 29us) server_negotiation.cc:407] Connection header received
0811 17:24:14.025594 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:14.025597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:14.025777 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:14.025933 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:24:14.026206 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:14.027015 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:14.027935 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:14.028224 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:14.028323 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:14.028678 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:24:14.028991 (+ 313us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:24:14.029102 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:14.029415 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 17:24:14.029482 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":85,"threads_started":1}
I20260811 17:24:14.029974 13074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:14.024971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48906 (local address 127.9.157.188:45039)
0811 17:24:14.025100 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:14.025104 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:14.025391 (+ 287us) server_negotiation.cc:407] Connection header received
0811 17:24:14.025576 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:14.025579 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:14.025634 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:14.025714 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:24:14.026020 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:14.026825 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:14.028165 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:14.028476 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:14.028600 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:14.029477 (+ 877us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:24:14.029554 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:24:14.029607 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:14.029797 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 17:24:14.029857 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 17:24:14.033232 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 12495
I20260811 17:24:14.054634 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 12613
I20260811 17:24:14.072285 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 12725
2026-08-11T17:24:14Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3547 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 17:24:14 dist-test-slave-mjjr krb5kdc[13091](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 17:24:14 dist-test-slave-mjjr krb5kdc[13091](info): set up 2 sockets
Aug 11 17:24:14 dist-test-slave-mjjr krb5kdc[13091](info): commencing operation
krb5kdc: starting...
W20260811 17:24:15.342672 9846 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/krb5kdc/test-user.keytab.
Aug 11 17:24:15 dist-test-slave-mjjr krb5kdc[13091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T17:24:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T17:24:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.157.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
Entry for principal kudu/127.9.157.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab.
I20260811 17:24:15.480823 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:42377
--webserver_interface=127.9.157.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:40775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:42377,127.9.157.189:44529,127.9.157.188:43545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:24:15.599702 13128 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 17:24:15.600030 13128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:24:15.600055 13128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:24:15.600095 13128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:24:15.604084 13128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:24:15.604146 13128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:24:15.604166 13128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:24:15.604183 13128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:24:15.604198 13128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:24:15.609726 13128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:40775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:42377,127.9.157.189:44529,127.9.157.188:43545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.190.keytab
--principal=kudu/127.9.157.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.190:42377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.157.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:24:15.610971 13128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:24:15.612109 13128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:24:15.621169 13137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:24:15.621168 13136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:24:15.621310 13128 server_base.cc:1034] running on GCE node
W20260811 17:24:15.621181 13141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:24:15.622087 13128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:24:15.623176 13128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:24:15.624382 13128 hybrid_clock.cc:648] HybridClock initialized: now 1786469055624364 us; error 34 us; skew 500 ppm
Aug 11 17:24:15 dist-test-slave-mjjr krb5kdc[13091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469055, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:24:15.627471 13128 init.cc:377] Logged in from keytab as kudu/127.9.157.190@KRBTEST.COM (short username kudu)
I20260811 17:24:15.628659 13128 webserver.cc:460] Webserver started at http://127.9.157.190:33583/ using document root <none> and password file <none>
I20260811 17:24:15.629278 13128 fs_manager.cc:359] Metadata directory not provided
I20260811 17:24:15.629345 13128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:24:15.629530 13128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:24:15.631860 13128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-0/data/instance:
uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99"
format_stamp: "Formatted at 2026-08-11 17:24:15 on dist-test-slave-mjjr"
server_key: "21b6579617ef85f0dd4427e2ce34de3f"
server_key_iv: "ccf5967ad6288f66dd560032a3385294"
server_key_version: "encryptionkey@0"
I20260811 17:24:15.632458 13128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-0/wal/instance:
uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99"
format_stamp: "Formatted at 2026-08-11 17:24:15 on dist-test-slave-mjjr"
server_key: "21b6579617ef85f0dd4427e2ce34de3f"
server_key_iv: "ccf5967ad6288f66dd560032a3385294"
server_key_version: "encryptionkey@0"
I20260811 17:24:15.636518 13128 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 17:24:15.639086 13153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:24:15.640492 13128 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 17:24:15.640641 13128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-0/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-0/wal
uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99"
format_stamp: "Formatted at 2026-08-11 17:24:15 on dist-test-slave-mjjr"
server_key: "21b6579617ef85f0dd4427e2ce34de3f"
server_key_iv: "ccf5967ad6288f66dd560032a3385294"
server_key_version: "encryptionkey@0"
I20260811 17:24:15.640761 13128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:24:15.653784 13128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:24:15.657186 13128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:24:15.657418 13128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:24:15.665216 13128 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.190:42377
I20260811 17:24:15.665529 13248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.190:42377 every 8 connection(s)
I20260811 17:24:15.666280 13128 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-0/data/info.pb
I20260811 17:24:15.669309 13250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:24:15.669689 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 13128
I20260811 17:24:15.669842 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-0/wal/instance
I20260811 17:24:15.670177 9846 external_mini_cluster.cc:1428] Setting key 0b9c7dbc3dc5afdaf76e0dc8e41ef415
I20260811 17:24:15.674221 13250 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42377 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.157.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.189@KRBTEST.COM" created.
Aug 11 17:24:15 dist-test-slave-mjjr krb5kdc[13091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469055, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:24:15.691579 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:15.676831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42377 (local address 127.0.0.1:50804)
0811 17:24:15.677385 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:15.677440 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 17:24:15.678408 (+ 968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:24:15.679302 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:24:15.679310 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:24:15.679767 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:24:15.680043 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:15.680057 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:15.681368 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:15.681371 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:15.682215 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:15.682225 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:15.683018 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:15.683026 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:15.683175 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:15.683962 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:24:15.683989 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:24:15.685966 (+ 1977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:24:15.688494 (+ 2528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:15.688503 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:15.688512 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:15.688734 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:15.689159 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:15.689161 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:15.689163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:15.689270 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:15.690134 (+ 864us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:24:15.690139 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:24:15.690268 (+ 129us) client_negotiation.cc:769] Sending connection context
0811 17:24:15.690414 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 17:24:15.690725 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":164,"threads_started":1}
I20260811 17:24:15.691586 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:15.677768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50804 (local address 127.9.157.190:42377)
0811 17:24:15.678145 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:15.678157 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:15.678206 (+ 49us) server_negotiation.cc:407] Connection header received
0811 17:24:15.678760 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:15.678770 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:15.679055 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:15.679278 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:15.680255 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:15.681175 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:15.682336 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:15.682791 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:15.682938 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:15.686145 (+ 3207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:15.686155 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:15.686169 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:15.686200 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:15.688380 (+ 2180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:15.688864 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:15.688874 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:15.688879 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:15.689051 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:15.689369 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:15.689376 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:15.689379 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:15.690000 (+ 621us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:15.690156 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:15.690550 (+ 394us) server_negotiation.cc:299] Negotiation successful
0811 17:24:15.690854 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":240,"thread_start_us":113,"threads_started":1}
I20260811 17:24:15.694530 13250 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.189" port: 44529 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:24:15.695374 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:15.695090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:44529 (local address 127.0.0.1:47818)
0811 17:24:15.695217 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:15.695297 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.189:44529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 17:24:15.695685 13155 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.157.189:44529: connect: Connection refused (error 111)
I20260811 17:24:15.696648 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:15.696419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:44529 (local address 127.0.0.1:47820)
0811 17:24:15.696541 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:15.696593 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.189:44529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 17:24:15.696805 13250 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.189:44529: Network error: Client connection negotiation failed: client connection to 127.9.157.189:44529: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.189.keytab.
Entry for principal kudu/127.9.157.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.189.keytab.
I20260811 17:24:15.702215 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.189:44529
--webserver_interface=127.9.157.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.189.keytab
--principal=kudu/127.9.157.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:40775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:42377,127.9.157.189:44529,127.9.157.188:43545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 17:24:15.746063 13250 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.189" port: 44529 } attempt: 1
I20260811 17:24:15.747051 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:15.746737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:44529 (local address 127.0.0.1:47826)
0811 17:24:15.746907 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:15.746969 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.189:44529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 17:24:15.748025 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:15.747792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:44529 (local address 127.0.0.1:47834)
0811 17:24:15.747925 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:15.747970 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.189:44529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 17:24:15.748170 13250 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.189:44529: Network error: Client connection negotiation failed: client connection to 127.9.157.189:44529: connect: Connection refused (error 111)
I20260811 17:24:15.816329 13250 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.189" port: 44529 } attempt: 2
I20260811 17:24:15.817315 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:15.817004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:44529 (local address 127.0.0.1:47842)
0811 17:24:15.817162 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:15.817228 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.189:44529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 17:24:15.818200 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:15.817963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:44529 (local address 127.0.0.1:47856)
0811 17:24:15.818080 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:15.818130 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.189:44529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 17:24:15.818431 13250 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.189:44529: Network error: Client connection negotiation failed: client connection to 127.9.157.189:44529: connect: Connection refused (error 111)
W20260811 17:24:15.823896 13257 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 17:24:15.824218 13257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:24:15.824298 13257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:24:15.824389 13257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:24:15.828784 13257 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:24:15.828888 13257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:24:15.828927 13257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:24:15.828956 13257 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:24:15.828979 13257 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:24:15.834805 13257 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:40775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:42377,127.9.157.189:44529,127.9.157.188:43545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.189.keytab
--principal=kudu/127.9.157.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.189:44529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.157.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:24:15.836086 13257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:24:15.837184 13257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:24:15.846407 13265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:24:15.846407 13267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:24:15.846546 13257 server_base.cc:1034] running on GCE node
W20260811 17:24:15.846436 13264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:24:15.847405 13257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:24:15.848654 13257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:24:15.849869 13257 hybrid_clock.cc:648] HybridClock initialized: now 1786469055849853 us; error 27 us; skew 500 ppm
Aug 11 17:24:15 dist-test-slave-mjjr krb5kdc[13091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469055, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:24:15.852854 13257 init.cc:377] Logged in from keytab as kudu/127.9.157.189@KRBTEST.COM (short username kudu)
I20260811 17:24:15.853776 13257 webserver.cc:460] Webserver started at http://127.9.157.189:45187/ using document root <none> and password file <none>
I20260811 17:24:15.854374 13257 fs_manager.cc:359] Metadata directory not provided
I20260811 17:24:15.854470 13257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:24:15.854717 13257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:24:15.856868 13257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-1/data/instance:
uuid: "656f43309a62496d9fcc5fb0a4cc419c"
format_stamp: "Formatted at 2026-08-11 17:24:15 on dist-test-slave-mjjr"
server_key: "084bd9b4fb8a4b7f38056c8e320cca0b"
server_key_iv: "ab63dde4e2b87901179fa59d8548fbf6"
server_key_version: "encryptionkey@0"
I20260811 17:24:15.857460 13257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-1/wal/instance:
uuid: "656f43309a62496d9fcc5fb0a4cc419c"
format_stamp: "Formatted at 2026-08-11 17:24:15 on dist-test-slave-mjjr"
server_key: "084bd9b4fb8a4b7f38056c8e320cca0b"
server_key_iv: "ab63dde4e2b87901179fa59d8548fbf6"
server_key_version: "encryptionkey@0"
I20260811 17:24:15.861331 13257 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 17:24:15.863828 13278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:24:15.865259 13257 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 17:24:15.865433 13257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-1/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-1/wal
uuid: "656f43309a62496d9fcc5fb0a4cc419c"
format_stamp: "Formatted at 2026-08-11 17:24:15 on dist-test-slave-mjjr"
server_key: "084bd9b4fb8a4b7f38056c8e320cca0b"
server_key_iv: "ab63dde4e2b87901179fa59d8548fbf6"
server_key_version: "encryptionkey@0"
I20260811 17:24:15.865578 13257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:24:15.896945 13257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:24:15.899935 13257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:24:15.900163 13257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:24:15.907610 13257 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.189:44529
I20260811 17:24:15.907963 13361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.189:44529 every 8 connection(s)
I20260811 17:24:15.908682 13257 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-1/data/info.pb
I20260811 17:24:15.909600 13250 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.189" port: 44529 } attempt: 3
I20260811 17:24:15.910884 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 13257
I20260811 17:24:15.911075 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-1/wal/instance
I20260811 17:24:15.911479 9846 external_mini_cluster.cc:1428] Setting key 2261f39ed1a06155122f46a41826e021
I20260811 17:24:15.912218 13363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:24:15.917456 13363 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42377 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 17:24:15 dist-test-slave-mjjr krb5kdc[13091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469055, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for kudu/127.9.157.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.157.188@KRBTEST.COM; defaulting to no policy
I20260811 17:24:15.924357 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:15.910169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:44529 (local address 127.0.0.1:47858)
0811 17:24:15.910341 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:15.910358 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:24:15.910458 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:24:15.913641 (+ 3183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:24:15.913645 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:24:15.913854 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:24:15.913997 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:15.914005 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:15.915295 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:15.915300 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:15.916026 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:15.916042 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:15.917030 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:15.917037 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:15.917193 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:15.917840 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:24:15.917863 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:24:15.919353 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:24:15.922286 (+ 2933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:15.922289 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:15.922291 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:15.922530 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:15.922889 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:15.922895 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:15.922896 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:15.923028 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:15.923772 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:24:15.923775 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:24:15.923885 (+ 110us) client_negotiation.cc:769] Sending connection context
0811 17:24:15.924010 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 17:24:15.924182 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":1}
I20260811 17:24:15.925598 13365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:15.910993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47858 (local address 127.9.157.189:44529)
0811 17:24:15.911493 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:15.911517 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:15.911617 (+ 100us) server_negotiation.cc:407] Connection header received
0811 17:24:15.912777 (+ 1160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:15.912805 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:15.913271 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:15.913659 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:15.914127 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:15.915136 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:15.916168 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:15.916876 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:15.917047 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:15.919641 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:15.919649 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:15.919695 (+ 46us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:15.919738 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:15.922134 (+ 2396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:15.922659 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:15.922664 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:15.922669 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:15.922784 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:15.923124 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:15.923128 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:15.923130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:15.923602 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:15.923801 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:15.924163 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 17:24:15.924468 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":150,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.157.188@KRBTEST.COM" created.
Aug 11 17:24:15 dist-test-slave-mjjr krb5kdc[13091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469055, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.189@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:24:15.928786 13250 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:24:15.929668 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:15.929299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:43545 (local address 127.0.0.1:42670)
0811 17:24:15.929504 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:15.929569 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:43545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260811 17:24:15.930521 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:15.930182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:43545 (local address 127.0.0.1:42676)
0811 17:24:15.930399 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:15.930453 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:43545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
W20260811 17:24:15.930785 13250 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.188:43545: Network error: Client connection negotiation failed: client connection to 127.9.157.188:43545: connect: Connection refused (error 111)
I20260811 17:24:15.932358 13371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:15.919660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42377 (local address 127.0.0.1:50812)
0811 17:24:15.920249 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:15.920299 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 17:24:15.920422 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:24:15.920806 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:24:15.920814 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:24:15.921258 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:24:15.921380 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:15.921386 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:15.922649 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:15.922654 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:15.923730 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:15.923742 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:15.924338 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:15.924341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:15.924618 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:15.925180 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:24:15.925192 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:24:15.927985 (+ 2793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:24:15.930287 (+ 2302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:15.930306 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:15.930316 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:15.930742 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:15.931084 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:15.931087 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:15.931090 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:15.931183 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:15.931746 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:24:15.931752 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:24:15.931891 (+ 139us) client_negotiation.cc:769] Sending connection context
0811 17:24:15.932019 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 17:24:15.932207 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":278,"threads_started":1}
I20260811 17:24:15.932514 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:15.919709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50812 (local address 127.9.157.190:42377)
0811 17:24:15.919843 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:15.919848 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:15.920384 (+ 536us) server_negotiation.cc:407] Connection header received
0811 17:24:15.920541 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:15.920545 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:15.920600 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:15.920642 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:15.921508 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:15.922499 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:15.923883 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:15.924172 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:15.924330 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:15.928161 (+ 3831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:15.928168 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:15.928175 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:15.928211 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:15.930151 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:15.930863 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:15.930869 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:15.930872 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:15.930979 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:15.931291 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:15.931297 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:15.931301 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:15.931577 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:15.931714 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:15.932151 (+ 437us) server_negotiation.cc:299] Negotiation successful
0811 17:24:15.932297 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":50}
I20260811 17:24:15.933594 13363 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.189" port: 44529 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 17:24:15 dist-test-slave-mjjr krb5kdc[13091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469055, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.189@KRBTEST.COM for kudu/127.9.157.189@KRBTEST.COM
I20260811 17:24:15.944684 13371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:15.934250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:44529 (local address 127.0.0.1:47864)
0811 17:24:15.934379 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:15.934397 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 17:24:15.934487 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:24:15.934765 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:24:15.934768 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:24:15.934924 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:24:15.935032 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:15.935037 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:15.936180 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:15.936183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:15.936695 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:15.936703 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:15.937370 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:15.937372 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:15.937451 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:15.938000 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:24:15.938011 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:24:15.941194 (+ 3183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:24:15.943030 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:15.943033 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:15.943035 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:15.943281 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:15.943545 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:15.943547 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:15.943549 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:15.943624 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:15.944106 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:24:15.944108 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:24:15.944241 (+ 133us) client_negotiation.cc:769] Sending connection context
0811 17:24:15.944353 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 17:24:15.944473 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 17:24:15.944779 13365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:15.934320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47864 (local address 127.9.157.189:44529)
0811 17:24:15.934468 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:15.934476 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:15.934494 (+ 18us) server_negotiation.cc:407] Connection header received
0811 17:24:15.934565 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:15.934571 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:15.934629 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:15.934727 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:15.935159 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:15.936063 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:15.936825 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:15.937233 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:15.937365 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:15.941358 (+ 3993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:15.941361 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:15.941364 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:15.941390 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:15.942915 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:15.943374 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:15.943378 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:15.943380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:15.943457 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:15.943794 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:15.943797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:15.943798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:15.944020 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:15.944130 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:15.944420 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 17:24:15.944552 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.157.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.188.keytab.
Entry for principal kudu/127.9.157.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.188.keytab.
I20260811 17:24:15.945655 13363 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:24:15.946503 13371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:15.946206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:43545 (local address 127.0.0.1:42678)
0811 17:24:15.946335 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:15.946433 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:43545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 17:24:15.946720 9846 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
/tmp/dist-test-tasklEMhqg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.188:43545
--webserver_interface=127.9.157.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.188.keytab
--principal=kudu/127.9.157.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.157.148:40775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.157.190:42377,127.9.157.189:44529,127.9.157.188:43545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 17:24:15.947293 13284 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.157.188:43545: connect: Connection refused (error 111)
I20260811 17:24:15.948431 13371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:15.948179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:43545 (local address 127.0.0.1:42682)
0811 17:24:15.948320 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:15.948372 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:43545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 17:24:15.948719 13363 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.188:43545: Network error: Client connection negotiation failed: client connection to 127.9.157.188:43545: connect: Connection refused (error 111)
I20260811 17:24:15.961994 13250 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 } attempt: 1
I20260811 17:24:15.963014 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:15.962661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:43545 (local address 127.0.0.1:42684)
0811 17:24:15.962837 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:15.962920 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:43545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 17:24:15.963871 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:15.963602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:43545 (local address 127.0.0.1:42688)
0811 17:24:15.963755 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:15.963798 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:43545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260811 17:24:15.964039 13250 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.188:43545: Network error: Client connection negotiation failed: client connection to 127.9.157.188:43545: connect: Connection refused (error 111)
I20260811 17:24:15.998085 13363 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 } attempt: 1
I20260811 17:24:15.999182 13371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:15.998818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:43545 (local address 127.0.0.1:42696)
0811 17:24:15.999018 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:15.999093 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:43545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 17:24:16.000176 13371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:15.999928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:43545 (local address 127.0.0.1:42700)
0811 17:24:16.000052 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:16.000114 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:43545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 17:24:16.000336 13363 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.188:43545: Network error: Client connection negotiation failed: client connection to 127.9.157.188:43545: connect: Connection refused (error 111)
I20260811 17:24:16.039240 13250 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 } attempt: 2
I20260811 17:24:16.040246 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:16.039930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:43545 (local address 127.0.0.1:42716)
0811 17:24:16.040096 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:16.040163 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:43545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 17:24:16.041049 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:16.040839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:43545 (local address 127.0.0.1:42728)
0811 17:24:16.040947 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:16.040993 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:43545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 17:24:16.041287 13250 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.188:43545: Network error: Client connection negotiation failed: client connection to 127.9.157.188:43545: connect: Connection refused (error 111)
I20260811 17:24:16.068529 13363 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 } attempt: 2
I20260811 17:24:16.069434 13371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:16.069118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:43545 (local address 127.0.0.1:42732)
0811 17:24:16.069271 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:16.069343 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:43545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 17:24:16.070302 13371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:16.070106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:43545 (local address 127.0.0.1:42744)
0811 17:24:16.070206 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:16.070250 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:43545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 17:24:16.070519 13363 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.188:43545: Network error: Client connection negotiation failed: client connection to 127.9.157.188:43545: connect: Connection refused (error 111)
W20260811 17:24:16.072110 13377 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 17:24:16.072430 13377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 17:24:16.072474 13377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 17:24:16.072549 13377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 17:24:16.076839 13377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 17:24:16.076896 13377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 17:24:16.076916 13377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 17:24:16.076957 13377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 17:24:16.076992 13377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 17:24:16.082495 13377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.157.148:40775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.157.190:42377,127.9.157.189:44529,127.9.157.188:43545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/krb5kdc/kudu_127.9.157.188.keytab
--principal=kudu/127.9.157.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.157.188:43545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.157.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 16:30:47 UTC on ccycloud.quasar-gzgosm.root.comops.site
build id 4580888
I20260811 17:24:16.083690 13377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 17:24:16.084836 13377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 17:24:16.093536 13387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:24:16.093606 13377 server_base.cc:1034] running on GCE node
W20260811 17:24:16.093564 13388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 17:24:16.093536 13391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 17:24:16.094410 13377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 17:24:16.095674 13377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 17:24:16.096887 13377 hybrid_clock.cc:648] HybridClock initialized: now 1786469056096868 us; error 33 us; skew 500 ppm
Aug 11 17:24:16 dist-test-slave-mjjr krb5kdc[13091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469056, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 17:24:16.099843 13377 init.cc:377] Logged in from keytab as kudu/127.9.157.188@KRBTEST.COM (short username kudu)
I20260811 17:24:16.100771 13377 webserver.cc:460] Webserver started at http://127.9.157.188:46213/ using document root <none> and password file <none>
I20260811 17:24:16.101359 13377 fs_manager.cc:359] Metadata directory not provided
I20260811 17:24:16.101451 13377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 17:24:16.101696 13377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 17:24:16.103950 13377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-2/data/instance:
uuid: "1135aa52eee74c679eaa179071c05a76"
format_stamp: "Formatted at 2026-08-11 17:24:16 on dist-test-slave-mjjr"
server_key: "2ad6f698e4e97a4de5d06d12bd6d297a"
server_key_iv: "601ab530bd0ee90107f8bd6c945c5e58"
server_key_version: "encryptionkey@0"
I20260811 17:24:16.104540 13377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-2/wal/instance:
uuid: "1135aa52eee74c679eaa179071c05a76"
format_stamp: "Formatted at 2026-08-11 17:24:16 on dist-test-slave-mjjr"
server_key: "2ad6f698e4e97a4de5d06d12bd6d297a"
server_key_iv: "601ab530bd0ee90107f8bd6c945c5e58"
server_key_version: "encryptionkey@0"
I20260811 17:24:16.108502 13377 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 17:24:16.110975 13402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 17:24:16.112267 13377 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 17:24:16.112404 13377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-2/data,/tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-2/wal
uuid: "1135aa52eee74c679eaa179071c05a76"
format_stamp: "Formatted at 2026-08-11 17:24:16 on dist-test-slave-mjjr"
server_key: "2ad6f698e4e97a4de5d06d12bd6d297a"
server_key_iv: "601ab530bd0ee90107f8bd6c945c5e58"
server_key_version: "encryptionkey@0"
I20260811 17:24:16.112522 13377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 17:24:16.135241 13377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 17:24:16.138820 13377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 17:24:16.139041 13377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 17:24:16.140480 13250 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 } attempt: 3
I20260811 17:24:16.141330 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:16.140987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:43545 (local address 127.0.0.1:42754)
0811 17:24:16.141152 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:16.141230 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:43545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 17:24:16.142118 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:16.141860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:43545 (local address 127.0.0.1:42756)
0811 17:24:16.141989 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:16.142044 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.157.188:43545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 17:24:16.142259 13250 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.157.188:43545: Network error: Client connection negotiation failed: client connection to 127.9.157.188:43545: connect: Connection refused (error 111)
I20260811 17:24:16.147264 13377 rpc_server.cc:307] RPC server started. Bound to: 127.9.157.188:43545
I20260811 17:24:16.147559 13483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.157.188:43545 every 8 connection(s)
I20260811 17:24:16.148301 13377 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-2/data/info.pb
I20260811 17:24:16.151181 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 17:24:16.155149 13485 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42377 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:24:16.155802 9846 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu as pid 13377
I20260811 17:24:16.155965 9846 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklEMhqg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786468743878666-9846-0/minicluster-data/master-2/wal/instance
I20260811 17:24:16.156280 9846 external_mini_cluster.cc:1428] Setting key 00fcdcb2cec35067cffa473897470350
I20260811 17:24:16.161724 13363 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 } attempt: 3
Aug 11 17:24:16 dist-test-slave-mjjr krb5kdc[13091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469056, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.188@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
Aug 11 17:24:16 dist-test-slave-mjjr krb5kdc[13091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.188@KRBTEST.COM
Aug 11 17:24:16 dist-test-slave-mjjr krb5kdc[13091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469055, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.189@KRBTEST.COM for kudu/127.9.157.188@KRBTEST.COM
I20260811 17:24:16.173236 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:16.157504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50818 (local address 127.9.157.190:42377)
0811 17:24:16.157631 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:16.157636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:16.158332 (+ 696us) server_negotiation.cc:407] Connection header received
0811 17:24:16.159191 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:16.159195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:16.159248 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:16.159338 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:16.160507 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.161295 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.162720 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.163074 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.163197 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:16.167012 (+ 3815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:16.167020 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:16.167022 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:16.167049 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:16.169373 (+ 2324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:16.170068 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:16.170072 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:16.170073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:16.170234 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:16.171845 (+ 1611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:16.171849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:16.171850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:16.172117 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:16.172191 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:16.172759 (+ 568us) server_negotiation.cc:299] Negotiation successful
0811 17:24:16.173064 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 17:24:16.173246 13371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:16.162225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:43545 (local address 127.0.0.1:42776)
0811 17:24:16.162451 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:16.162467 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 17:24:16.162563 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:24:16.163061 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:24:16.163065 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:24:16.163268 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:24:16.163407 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:16.163416 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.164515 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.164519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:16.165036 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:16.165044 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.165516 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.165521 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:16.165641 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:16.166160 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:24:16.166172 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:24:16.167689 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:24:16.171496 (+ 3807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:16.171500 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:16.171505 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:16.171850 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:16.172165 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:16.172168 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:16.172170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:16.172266 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:16.172750 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:24:16.172753 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:24:16.172861 (+ 108us) client_negotiation.cc:769] Sending connection context
0811 17:24:16.172958 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 17:24:16.173107 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
I20260811 17:24:16.174338 13491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:16.157664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42760 (local address 127.9.157.188:43545)
0811 17:24:16.158124 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:16.158141 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:16.158176 (+ 35us) server_negotiation.cc:407] Connection header received
0811 17:24:16.158918 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:16.158938 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:16.159195 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:16.159384 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:16.160330 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.161579 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.162757 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.163285 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.163439 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:16.166916 (+ 3477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:16.166927 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:16.166945 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:16.166975 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:16.169189 (+ 2214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:16.169758 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:16.169798 (+ 40us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:16.169802 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:16.170072 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:16.170590 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:16.170596 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:16.170600 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:16.171187 (+ 587us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:16.171400 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:16.172311 (+ 911us) server_negotiation.cc:299] Negotiation successful
0811 17:24:16.172660 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":323,"spinlock_wait_cycles":21760,"thread_start_us":181,"threads_started":1}
I20260811 17:24:16.174340 13489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:16.157665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.190:42377 (local address 127.0.0.1:50818)
0811 17:24:16.158166 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:16.158217 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 17:24:16.158933 (+ 716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:24:16.159412 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:24:16.159418 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:24:16.159963 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:24:16.160357 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:16.160371 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.161429 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.161432 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:16.162582 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:16.162593 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.163233 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.163237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:16.163438 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:16.164312 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:24:16.164335 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:24:16.166801 (+ 2466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:24:16.169554 (+ 2753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:16.169585 (+ 31us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:16.169596 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:16.169951 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:16.170347 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:16.170357 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:16.170359 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:16.170518 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:16.172279 (+ 1761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:24:16.172286 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:24:16.172427 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 17:24:16.173406 (+ 979us) client_negotiation.cc:240] Negotiation successful
0811 17:24:16.173612 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":37,"spinlock_wait_cycles":1792,"thread_start_us":160,"threads_started":1}
I20260811 17:24:16.174340 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:16.162262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42776 (local address 127.9.157.188:43545)
0811 17:24:16.162809 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:16.162817 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:16.162829 (+ 12us) server_negotiation.cc:407] Connection header received
0811 17:24:16.162859 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:16.162862 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:16.162911 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:16.162996 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:16.163540 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.164390 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.165160 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.165396 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.165505 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:16.167841 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:16.167845 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:16.167847 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:16.167872 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:16.171308 (+ 3436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:16.171968 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:16.171971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:16.171973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:16.172049 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:16.172437 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:16.172440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:16.172442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:16.172641 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:16.173578 (+ 937us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:16.173648 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 17:24:16.173740 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":480,"thread_start_us":153,"threads_started":1}
I20260811 17:24:16.180161 13363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 656f43309a62496d9fcc5fb0a4cc419c: Bootstrap starting.
I20260811 17:24:16.183467 13363 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 656f43309a62496d9fcc5fb0a4cc419c: Neither blocks nor log segments found. Creating new log.
I20260811 17:24:16.183635 13485 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.189" port: 44529 } has no permanent_uuid. Determining permanent_uuid...
I20260811 17:24:16.184825 13363 log.cc:824] T 00000000000000000000000000000000 P 656f43309a62496d9fcc5fb0a4cc419c: Log is configured to *not* fsync() on all Append() calls
Aug 11 17:24:16 dist-test-slave-mjjr krb5kdc[13091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.190@KRBTEST.COM
I20260811 17:24:16.188808 13363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 656f43309a62496d9fcc5fb0a4cc419c: No bootstrap required, opened a new log
Aug 11 17:24:16 dist-test-slave-mjjr krb5kdc[13091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469056, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.188@KRBTEST.COM for kudu/127.9.157.189@KRBTEST.COM
I20260811 17:24:16.192590 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:16.178966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50828 (local address 127.9.157.190:42377)
0811 17:24:16.179581 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:16.179586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:16.179945 (+ 359us) server_negotiation.cc:407] Connection header received
0811 17:24:16.179991 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:16.179994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:16.180045 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:16.180115 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:16.181035 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.181762 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.182846 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.183174 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.183294 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:16.187967 (+ 4673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:16.187974 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:16.187977 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:16.188011 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:16.190226 (+ 2215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:16.190820 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:16.190824 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:16.190827 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:16.190929 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:16.191294 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:16.191297 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:16.191299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:16.191527 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:16.191751 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:16.192268 (+ 517us) server_negotiation.cc:299] Negotiation successful
0811 17:24:16.192400 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516}
I20260811 17:24:16.194577 13363 raft_consensus.cc:348] T 00000000000000000000000000000000 P 656f43309a62496d9fcc5fb0a4cc419c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42377 } } peers { permanent_uuid: "656f43309a62496d9fcc5fb0a4cc419c" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 44529 } } peers { permanent_uuid: "1135aa52eee74c679eaa179071c05a76" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 } }
I20260811 17:24:16.194846 13363 raft_consensus.cc:729] T 00000000000000000000000000000000 P 656f43309a62496d9fcc5fb0a4cc419c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 656f43309a62496d9fcc5fb0a4cc419c, State: Initialized, Role: FOLLOWER
I20260811 17:24:16.195232 13363 consensus_queue.cc:260] T 00000000000000000000000000000000 P 656f43309a62496d9fcc5fb0a4cc419c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42377 } } peers { permanent_uuid: "656f43309a62496d9fcc5fb0a4cc419c" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 44529 } } peers { permanent_uuid: "1135aa52eee74c679eaa179071c05a76" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 } }
I20260811 17:24:16.195822 13495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 656f43309a62496d9fcc5fb0a4cc419c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42377 } } peers { permanent_uuid: "656f43309a62496d9fcc5fb0a4cc419c" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 44529 } } peers { permanent_uuid: "1135aa52eee74c679eaa179071c05a76" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 } } }
I20260811 17:24:16.196158 13365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:16.184696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47876 (local address 127.9.157.189:44529)
0811 17:24:16.184923 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:16.184929 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:16.184945 (+ 16us) server_negotiation.cc:407] Connection header received
0811 17:24:16.185016 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:16.185019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:16.185097 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:16.185223 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:16.185760 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.186643 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.188173 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.188587 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.188679 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:16.191509 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:16.191513 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:16.191518 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:16.191552 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:16.193636 (+ 2084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:16.194227 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:16.194232 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:16.194236 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:16.194332 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:16.194693 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:16.194698 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:16.194699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:16.194979 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:16.195071 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:16.195832 (+ 761us) server_negotiation.cc:299] Negotiation successful
0811 17:24:16.195972 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 17:24:16.195984 13495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 656f43309a62496d9fcc5fb0a4cc419c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:24:16.196684 13489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:16.184345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.189:44529 (local address 127.0.0.1:47876)
0811 17:24:16.184699 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:16.184716 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:24:16.184813 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:24:16.185250 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:24:16.185255 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:24:16.185469 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:24:16.185605 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:16.185616 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.186820 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.186825 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:16.188003 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:16.188020 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.188972 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.188980 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:16.189117 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:16.189571 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:24:16.189593 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:24:16.191320 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:24:16.193773 (+ 2453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:16.193779 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:16.193783 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:16.194090 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:16.194469 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:16.194474 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:16.194478 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:16.194579 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:16.195282 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:24:16.195286 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:24:16.195537 (+ 251us) client_negotiation.cc:769] Sending connection context
0811 17:24:16.196083 (+ 546us) client_negotiation.cc:240] Negotiation successful
0811 17:24:16.196460 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246}
I20260811 17:24:16.196851 13363 sys_catalog.cc:564] T 00000000000000000000000000000000 P 656f43309a62496d9fcc5fb0a4cc419c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:24:16.197682 13485 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 17:24:16 dist-test-slave-mjjr krb5kdc[13091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.157.189@KRBTEST.COM
Aug 11 17:24:16 dist-test-slave-mjjr krb5kdc[13091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469056, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.188@KRBTEST.COM for kudu/127.9.157.188@KRBTEST.COM
W20260811 17:24:16.204017 13516 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 656f43309a62496d9fcc5fb0a4cc419c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:24:16.204097 13516 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:24:16.206287 13494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:16.194482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47884 (local address 127.9.157.189:44529)
0811 17:24:16.194814 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:16.194824 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:16.194838 (+ 14us) server_negotiation.cc:407] Connection header received
0811 17:24:16.195056 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:16.195059 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:16.195107 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:16.195179 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:16.196128 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.196856 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.198013 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.198374 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.198477 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:16.201683 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:16.201687 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:16.201693 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:16.201721 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:16.203809 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:16.204328 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:16.204333 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:16.204336 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:16.204434 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:16.204810 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:16.204814 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:16.204816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:16.205073 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:16.205275 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:16.205969 (+ 694us) server_negotiation.cc:299] Negotiation successful
0811 17:24:16.206098 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":117,"threads_started":1}
I20260811 17:24:16.202466 13257 master_runner.cc:186] Error getting master registration for 127.9.157.190:42377: OK, instance_id { permanent_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" instance_seqno: 1786469055663068 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 17:24:16.207058 13257 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 17:24:16.208433 13489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:16.198284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:43545 (local address 127.0.0.1:42792)
0811 17:24:16.198483 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:16.198500 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:24:16.198597 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:24:16.198838 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:24:16.198843 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:24:16.199076 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:24:16.199204 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:16.199212 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.200330 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.200333 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:16.201048 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:16.201059 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.201769 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.201772 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:16.201837 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:16.202379 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:24:16.202393 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:24:16.203914 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:24:16.206032 (+ 2118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:16.206035 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:16.206039 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:16.206306 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:16.206945 (+ 639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:16.206950 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:16.206953 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:16.207201 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:16.207822 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:24:16.207826 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:24:16.207939 (+ 113us) client_negotiation.cc:769] Sending connection context
0811 17:24:16.208053 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 17:24:16.208214 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
I20260811 17:24:16.209375 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:16.198336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42792 (local address 127.9.157.188:43545)
0811 17:24:16.198472 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:16.198477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:16.198554 (+ 77us) server_negotiation.cc:407] Connection header received
0811 17:24:16.198681 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:16.198684 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:16.198729 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:16.198820 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:16.199345 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.200212 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.201245 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.201568 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.201669 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:16.204052 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:16.204057 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:16.204059 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:16.204083 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:16.205903 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:16.206725 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:16.206729 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:16.206732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:16.206828 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:16.207315 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:16.207320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:16.207323 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:16.207674 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:16.207782 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:16.209030 (+ 1248us) server_negotiation.cc:299] Negotiation successful
0811 17:24:16.209178 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 17:24:16.209682 9846 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 17:24:16.211082 13196 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50828
I20260811 17:24:16.213310 13485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1135aa52eee74c679eaa179071c05a76: Bootstrap starting.
I20260811 17:24:16.216348 13485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1135aa52eee74c679eaa179071c05a76: Neither blocks nor log segments found. Creating new log.
I20260811 17:24:16.217308 13485 log.cc:824] T 00000000000000000000000000000000 P 1135aa52eee74c679eaa179071c05a76: Log is configured to *not* fsync() on all Append() calls
I20260811 17:24:16.220026 13485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1135aa52eee74c679eaa179071c05a76: No bootstrap required, opened a new log
I20260811 17:24:16.224572 13485 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1135aa52eee74c679eaa179071c05a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42377 } } peers { permanent_uuid: "656f43309a62496d9fcc5fb0a4cc419c" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 44529 } } peers { permanent_uuid: "1135aa52eee74c679eaa179071c05a76" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 } }
I20260811 17:24:16.224870 13485 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1135aa52eee74c679eaa179071c05a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1135aa52eee74c679eaa179071c05a76, State: Initialized, Role: FOLLOWER
I20260811 17:24:16.225243 13485 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1135aa52eee74c679eaa179071c05a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42377 } } peers { permanent_uuid: "656f43309a62496d9fcc5fb0a4cc419c" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 44529 } } peers { permanent_uuid: "1135aa52eee74c679eaa179071c05a76" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 } }
I20260811 17:24:16.225888 13521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1135aa52eee74c679eaa179071c05a76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42377 } } peers { permanent_uuid: "656f43309a62496d9fcc5fb0a4cc419c" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 44529 } } peers { permanent_uuid: "1135aa52eee74c679eaa179071c05a76" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 } } }
I20260811 17:24:16.226189 13521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1135aa52eee74c679eaa179071c05a76 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:24:16.226835 13485 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1135aa52eee74c679eaa179071c05a76 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 17:24:16.232591 13544 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1135aa52eee74c679eaa179071c05a76: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 17:24:16.232734 13544 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 17:24:16.230764 13377 master_runner.cc:186] Error getting master registration for 127.9.157.190:42377: OK, instance_id { permanent_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" instance_seqno: 1786469055663068 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 17:24:16.234313 13377 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 17:24:16.306419 13250 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 } attempt: 4
Aug 11 17:24:16 dist-test-slave-mjjr krb5kdc[13091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786469055, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.157.190@KRBTEST.COM for kudu/127.9.157.188@KRBTEST.COM
I20260811 17:24:16.318881 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:16.307174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.157.188:43545 (local address 127.0.0.1:42794)
0811 17:24:16.307322 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 17:24:16.307339 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 17:24:16.307463 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 17:24:16.307985 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 17:24:16.307990 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 17:24:16.308258 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0811 17:24:16.308407 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:16.308417 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.309722 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.309727 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:16.310798 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 17:24:16.310815 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.311697 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.311703 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 17:24:16.311821 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:16.312446 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 17:24:16.312466 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 17:24:16.314345 (+ 1879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 17:24:16.316716 (+ 2371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:16.316720 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:16.316725 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:16.317101 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:16.317485 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 17:24:16.317490 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 17:24:16.317494 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 17:24:16.317600 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 17:24:16.318158 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 17:24:16.318163 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 17:24:16.318293 (+ 130us) client_negotiation.cc:769] Sending connection context
0811 17:24:16.318420 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 17:24:16.318610 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 17:24:16.319569 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:16.307624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42794 (local address 127.9.157.188:43545)
0811 17:24:16.307765 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:16.307770 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:16.307785 (+ 15us) server_negotiation.cc:407] Connection header received
0811 17:24:16.307833 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:16.307836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:16.307884 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:16.307955 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:16.308590 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.309548 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.310982 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.311393 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.311506 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:16.314545 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:16.314552 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:16.314557 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:16.314592 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:16.316555 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:16.317248 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:16.317253 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:16.317258 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:16.317359 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:16.317721 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:16.317726 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:16.317728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:16.318005 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:16.318106 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:16.319209 (+ 1103us) server_negotiation.cc:299] Negotiation successful
0811 17:24:16.319358 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 17:24:16.329860 13250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8edd1e7fb32469e89bf9f89a9f8bf99: Bootstrap starting.
I20260811 17:24:16.332953 13250 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8edd1e7fb32469e89bf9f89a9f8bf99: Neither blocks nor log segments found. Creating new log.
I20260811 17:24:16.333933 13250 log.cc:824] T 00000000000000000000000000000000 P b8edd1e7fb32469e89bf9f89a9f8bf99: Log is configured to *not* fsync() on all Append() calls
I20260811 17:24:16.336570 13250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8edd1e7fb32469e89bf9f89a9f8bf99: No bootstrap required, opened a new log
I20260811 17:24:16.340166 13250 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8edd1e7fb32469e89bf9f89a9f8bf99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42377 } } peers { permanent_uuid: "656f43309a62496d9fcc5fb0a4cc419c" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 44529 } } peers { permanent_uuid: "1135aa52eee74c679eaa179071c05a76" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 } }
I20260811 17:24:16.340435 13250 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8edd1e7fb32469e89bf9f89a9f8bf99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8edd1e7fb32469e89bf9f89a9f8bf99, State: Initialized, Role: FOLLOWER
I20260811 17:24:16.340823 13250 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8edd1e7fb32469e89bf9f89a9f8bf99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42377 } } peers { permanent_uuid: "656f43309a62496d9fcc5fb0a4cc419c" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 44529 } } peers { permanent_uuid: "1135aa52eee74c679eaa179071c05a76" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 } }
I20260811 17:24:16.342478 13250 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8edd1e7fb32469e89bf9f89a9f8bf99 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 17:24:16.342617 13196 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8edd1e7fb32469e89bf9f89a9f8bf99 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 17:24:16.342765 13196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8edd1e7fb32469e89bf9f89a9f8bf99 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:24:16.344336 13196 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8edd1e7fb32469e89bf9f89a9f8bf99 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42377 } } peers { permanent_uuid: "656f43309a62496d9fcc5fb0a4cc419c" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 44529 } } peers { permanent_uuid: "1135aa52eee74c679eaa179071c05a76" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 } }
I20260811 17:24:16.345533 13196 leader_election.cc:288] T 00000000000000000000000000000000 P b8edd1e7fb32469e89bf9f89a9f8bf99 [CANDIDATE]: Term 1 election: Requested vote from peers 656f43309a62496d9fcc5fb0a4cc419c (127.9.157.189:44529), 1135aa52eee74c679eaa179071c05a76 (127.9.157.188:43545)
I20260811 17:24:16.346501 13320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "656f43309a62496d9fcc5fb0a4cc419c"
I20260811 17:24:16.346783 13320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 656f43309a62496d9fcc5fb0a4cc419c [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:24:16.348421 13320 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 656f43309a62496d9fcc5fb0a4cc419c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8edd1e7fb32469e89bf9f89a9f8bf99 in term 1.
I20260811 17:24:16.349653 13441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1135aa52eee74c679eaa179071c05a76"
I20260811 17:24:16.349939 13441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1135aa52eee74c679eaa179071c05a76 [term 0 FOLLOWER]: Advancing to term 1
I20260811 17:24:16.351457 13441 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1135aa52eee74c679eaa179071c05a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8edd1e7fb32469e89bf9f89a9f8bf99 in term 1.
I20260811 17:24:16.352015 13155 leader_election.cc:302] T 00000000000000000000000000000000 P b8edd1e7fb32469e89bf9f89a9f8bf99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 656f43309a62496d9fcc5fb0a4cc419c, b8edd1e7fb32469e89bf9f89a9f8bf99; no voters:
I20260811 17:24:16.351987 13549 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8edd1e7fb32469e89bf9f89a9f8bf99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42377 } } peers { permanent_uuid: "656f43309a62496d9fcc5fb0a4cc419c" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 44529 } } peers { permanent_uuid: "1135aa52eee74c679eaa179071c05a76" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 } } }
I20260811 17:24:16.352550 13549 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8edd1e7fb32469e89bf9f89a9f8bf99 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:24:16.352639 13549 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8edd1e7fb32469e89bf9f89a9f8bf99 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 17:24:16.352710 13549 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8edd1e7fb32469e89bf9f89a9f8bf99 [term 1 LEADER]: Becoming Leader. State: Replica: b8edd1e7fb32469e89bf9f89a9f8bf99, State: Running, Role: LEADER
I20260811 17:24:16.353040 13549 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8edd1e7fb32469e89bf9f89a9f8bf99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42377 } } peers { permanent_uuid: "656f43309a62496d9fcc5fb0a4cc419c" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 44529 } } peers { permanent_uuid: "1135aa52eee74c679eaa179071c05a76" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 } }
I20260811 17:24:16.355420 13549 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8edd1e7fb32469e89bf9f89a9f8bf99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8edd1e7fb32469e89bf9f89a9f8bf99. Latest consensus state: current_term: 1 leader_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42377 } } peers { permanent_uuid: "656f43309a62496d9fcc5fb0a4cc419c" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 44529 } } peers { permanent_uuid: "1135aa52eee74c679eaa179071c05a76" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 } } }
I20260811 17:24:16.355538 13549 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8edd1e7fb32469e89bf9f89a9f8bf99 [sys.catalog]: This master's current role is: LEADER
I20260811 17:24:16.358647 13128 master_runner.cc:263] No leader master found from master b8edd1e7fb32469e89bf9f89a9f8bf99
I20260811 17:24:16.358729 13128 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 17:24:16.365393 13582 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 17:24:16.378284 13582 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 17:24:16.385670 13494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:16.356219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47892 (local address 127.9.157.189:44529)
0811 17:24:16.356945 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:16.356952 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:16.367694 (+ 10742us) server_negotiation.cc:407] Connection header received
0811 17:24:16.367763 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:16.367765 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:16.367829 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:16.367942 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:16.372065 (+ 4123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.372953 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.375316 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.375728 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.375859 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:16.379680 (+ 3821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:16.379685 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:16.379688 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:16.379726 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:16.383547 (+ 3821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:16.384126 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:16.384130 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:16.384132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:16.384234 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:16.384555 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:16.384559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:16.384561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:16.384801 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:16.384959 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:16.385343 (+ 384us) server_negotiation.cc:299] Negotiation successful
0811 17:24:16.385492 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632}
I20260811 17:24:16.389005 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:16.357195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50844 (local address 127.9.157.190:42377)
0811 17:24:16.357393 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:16.357399 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:16.360260 (+ 2861us) server_negotiation.cc:407] Connection header received
0811 17:24:16.360426 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:16.360430 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:16.360481 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:16.360536 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:16.361631 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.362397 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.363833 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.364239 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.364338 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:16.373598 (+ 9260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:16.373602 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:16.373605 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:16.373642 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:16.382312 (+ 8670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:16.387102 (+ 4790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:16.387107 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:16.387108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:16.387233 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:16.387925 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:16.387929 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:16.387931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:16.388180 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:16.388307 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:16.388691 (+ 384us) server_negotiation.cc:299] Negotiation successful
0811 17:24:16.388832 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
W20260811 17:24:16.390651 13412 tablet.cc:2369] T 00000000000000000000000000000000 P 1135aa52eee74c679eaa179071c05a76: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:24:16.395120 13441 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1135aa52eee74c679eaa179071c05a76 [term 1 FOLLOWER]: Refusing update from remote peer b8edd1e7fb32469e89bf9f89a9f8bf99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:24:16.395120 13320 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 656f43309a62496d9fcc5fb0a4cc419c [term 1 FOLLOWER]: Refusing update from remote peer b8edd1e7fb32469e89bf9f89a9f8bf99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 17:24:16.396091 13549 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b8edd1e7fb32469e89bf9f89a9f8bf99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1135aa52eee74c679eaa179071c05a76" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:24:16.396555 13549 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b8edd1e7fb32469e89bf9f89a9f8bf99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "656f43309a62496d9fcc5fb0a4cc419c" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 44529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 17:24:16.401674 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:16.359152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42810 (local address 127.9.157.188:43545)
0811 17:24:16.359275 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:16.359281 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:16.386327 (+ 27046us) server_negotiation.cc:407] Connection header received
0811 17:24:16.386484 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:16.386488 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:16.386557 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:16.386641 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:16.387967 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.388744 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.389971 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:16.390307 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:16.390384 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:16.396828 (+ 6444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 17:24:16.396832 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 17:24:16.396835 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 17:24:16.396881 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 17:24:16.399253 (+ 2372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:16.399955 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:16.399960 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:16.399963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:16.400064 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 17:24:16.400423 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 17:24:16.400429 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 17:24:16.400431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 17:24:16.400715 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 17:24:16.400868 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:16.401295 (+ 427us) server_negotiation.cc:299] Negotiation successful
0811 17:24:16.401450 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 17:24:16.403759 13495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 656f43309a62496d9fcc5fb0a4cc419c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8edd1e7fb32469e89bf9f89a9f8bf99. Latest consensus state: current_term: 1 leader_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42377 } } peers { permanent_uuid: "656f43309a62496d9fcc5fb0a4cc419c" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 44529 } } peers { permanent_uuid: "1135aa52eee74c679eaa179071c05a76" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 } } }
I20260811 17:24:16.403901 13495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 656f43309a62496d9fcc5fb0a4cc419c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:24:16.404059 13521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1135aa52eee74c679eaa179071c05a76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8edd1e7fb32469e89bf9f89a9f8bf99. Latest consensus state: current_term: 1 leader_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42377 } } peers { permanent_uuid: "656f43309a62496d9fcc5fb0a4cc419c" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 44529 } } peers { permanent_uuid: "1135aa52eee74c679eaa179071c05a76" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 } } }
I20260811 17:24:16.404166 13521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1135aa52eee74c679eaa179071c05a76 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 17:24:16.404366 13288 tablet.cc:2369] T 00000000000000000000000000000000 P 656f43309a62496d9fcc5fb0a4cc419c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 17:24:16.409646 13162 tablet.cc:2369] T 00000000000000000000000000000000 P b8edd1e7fb32469e89bf9f89a9f8bf99: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 17:24:16.417994 13578 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8edd1e7fb32469e89bf9f89a9f8bf99 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42377 } } peers { permanent_uuid: "656f43309a62496d9fcc5fb0a4cc419c" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 44529 } } peers { permanent_uuid: "1135aa52eee74c679eaa179071c05a76" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 } } }
I20260811 17:24:16.418159 13578 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8edd1e7fb32469e89bf9f89a9f8bf99 [sys.catalog]: This master's current role is: LEADER
I20260811 17:24:16.420516 13549 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8edd1e7fb32469e89bf9f89a9f8bf99 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42377 } } peers { permanent_uuid: "656f43309a62496d9fcc5fb0a4cc419c" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 44529 } } peers { permanent_uuid: "1135aa52eee74c679eaa179071c05a76" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 } } }
I20260811 17:24:16.420641 13549 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8edd1e7fb32469e89bf9f89a9f8bf99 [sys.catalog]: This master's current role is: LEADER
I20260811 17:24:16.420842 13599 mvcc.cc:205] Tried to move back new op lower bound from 7317377254984294400 to 7317377254827372544. Current Snapshot: MvccSnapshot[applied={T|T < 7317377254984294400}]
I20260811 17:24:16.421176 13521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1135aa52eee74c679eaa179071c05a76 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42377 } } peers { permanent_uuid: "656f43309a62496d9fcc5fb0a4cc419c" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 44529 } } peers { permanent_uuid: "1135aa52eee74c679eaa179071c05a76" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 } } }
I20260811 17:24:16.421293 13521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1135aa52eee74c679eaa179071c05a76 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:24:16.421463 13495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 656f43309a62496d9fcc5fb0a4cc419c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8edd1e7fb32469e89bf9f89a9f8bf99" member_type: VOTER last_known_addr { host: "127.9.157.190" port: 42377 } } peers { permanent_uuid: "656f43309a62496d9fcc5fb0a4cc419c" member_type: VOTER last_known_addr { host: "127.9.157.189" port: 44529 } } peers { permanent_uuid: "1135aa52eee74c679eaa179071c05a76" member_type: VOTER last_known_addr { host: "127.9.157.188" port: 43545 } } }
I20260811 17:24:16.421586 13495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 656f43309a62496d9fcc5fb0a4cc419c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 17:24:16.421835 13595 mvcc.cc:205] Tried to move back new op lower bound from 7317377254984294400 to 7317377254827372544. Current Snapshot: MvccSnapshot[applied={T|T < 7317377254984294400}]
I20260811 17:24:16.425464 13582 catalog_manager.cc:1347] Generated new cluster ID: d4b08014c31b40488c8d9ad7156ff1f0
I20260811 17:24:16.425544 13582 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 17:24:16.449498 13582 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 17:24:16.450893 13582 catalog_manager.cc:1504] Loading token signing keys...
I20260811 17:24:16.463755 13582 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8edd1e7fb32469e89bf9f89a9f8bf99: Generated new TSK 0
I20260811 17:24:16.464733 13582 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 17:24:16.513795 13494 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:24:16.507986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47900 (local address 127.9.157.189:44529)
0811 17:24:16.508164 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:16.508170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:16.512048 (+ 3878us) server_negotiation.cc:407] Connection header received
0811 17:24:16.512907 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:16.512912 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:16.512967 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:16.513026 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:16.513654 (+ 628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47900: BlockingRecv error: recv got EOF from 127.0.0.1:47900 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 17:24:16.513795 13492 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:24:16.512089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42824 (local address 127.9.157.188:43545)
0811 17:24:16.512235 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:16.512240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:16.512779 (+ 539us) server_negotiation.cc:407] Connection header received
0811 17:24:16.512909 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:16.512912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:16.512967 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:16.513051 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:16.513647 (+ 596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42824: BlockingRecv error: recv got EOF from 127.0.0.1:42824 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 17:24:16.514570 13254 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:24:16.512360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50858 (local address 127.9.157.190:42377)
0811 17:24:16.512526 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:16.512532 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:16.513713 (+ 1181us) server_negotiation.cc:407] Connection header received
0811 17:24:16.513825 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:16.513832 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:16.513910 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:16.513990 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:16.514434 (+ 444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50858: BlockingRecv error: recv got EOF from 127.0.0.1:50858 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 17:24:16.518177 13494 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:24:16.516553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47916 (local address 127.9.157.189:44529)
0811 17:24:16.516685 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:16.516689 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:16.517206 (+ 517us) server_negotiation.cc:407] Connection header received
0811 17:24:16.517445 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:16.517448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:16.517530 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:16.517582 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:16.518080 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47916: BlockingRecv error: recv got EOF from 127.0.0.1:47916 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 17:24:16.541759 13494 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:24:16.529162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47922 (local address 127.9.157.189:44529)
0811 17:24:16.529291 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:16.529296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:16.540477 (+ 11181us) server_negotiation.cc:407] Connection header received
0811 17:24:16.540743 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:16.540749 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:16.540824 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:16.540901 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:16.541601 (+ 700us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47922: BlockingRecv error: recv got EOF from 127.0.0.1:47922 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 17:24:16.548180 13494 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:24:16.546724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47926 (local address 127.9.157.189:44529)
0811 17:24:16.546859 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:16.546864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:16.547010 (+ 146us) server_negotiation.cc:407] Connection header received
0811 17:24:16.547192 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:16.547197 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:16.547271 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:16.547357 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:16.548032 (+ 675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47926: BlockingRecv error: recv got EOF from 127.0.0.1:47926 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 17:24:16.556385 13494 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:24:16.554936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47936 (local address 127.9.157.189:44529)
0811 17:24:16.555110 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:16.555115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:16.555317 (+ 202us) server_negotiation.cc:407] Connection header received
0811 17:24:16.555544 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:16.555550 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:16.555626 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:16.555736 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:16.556238 (+ 502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47936: BlockingRecv error: recv got EOF from 127.0.0.1:47936 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
W20260811 17:24:16.568471 13494 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:24:16.567425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47946 (local address 127.9.157.189:44529)
0811 17:24:16.567551 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:16.567555 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:16.567572 (+ 17us) server_negotiation.cc:407] Connection header received
0811 17:24:16.567662 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:16.567666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:16.567728 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:16.567827 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:16.568349 (+ 522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47946: BlockingRecv error: recv got EOF from 127.0.0.1:47946 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 17:24:16.589171 13494 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:24:16.587509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47948 (local address 127.9.157.189:44529)
0811 17:24:16.587696 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:16.587701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:16.587977 (+ 276us) server_negotiation.cc:407] Connection header received
0811 17:24:16.588154 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:16.588161 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:16.588235 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:16.588329 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:16.589014 (+ 685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47948: BlockingRecv error: recv got EOF from 127.0.0.1:47948 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 17:24:16.625590 13494 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:24:16.624066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47964 (local address 127.9.157.189:44529)
0811 17:24:16.624215 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:16.624220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:16.624525 (+ 305us) server_negotiation.cc:407] Connection header received
0811 17:24:16.624744 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:16.624748 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:16.624808 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:16.624872 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:16.625455 (+ 583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47964: BlockingRecv error: recv got EOF from 127.0.0.1:47964 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 17:24:16.693588 13494 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:24:16.692285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47974 (local address 127.9.157.189:44529)
0811 17:24:16.692449 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:16.692454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:16.692645 (+ 191us) server_negotiation.cc:407] Connection header received
0811 17:24:16.692825 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:16.692834 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:16.692890 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:16.692984 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:16.693482 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47974: BlockingRecv error: recv got EOF from 127.0.0.1:47974 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 17:24:16.825314 13494 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:24:16.824055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47990 (local address 127.9.157.189:44529)
0811 17:24:16.824212 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:16.824216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:16.824379 (+ 163us) server_negotiation.cc:407] Connection header received
0811 17:24:16.824498 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:16.824502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:16.824553 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:16.824641 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:16.825166 (+ 525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47990: BlockingRecv error: recv got EOF from 127.0.0.1:47990 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 17:24:17.085263 13494 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 17:24:17.084021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48000 (local address 127.9.157.189:44529)
0811 17:24:17.084211 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:17.084216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:17.084237 (+ 21us) server_negotiation.cc:407] Connection header received
0811 17:24:17.084440 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:17.084443 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:17.084500 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:17.084584 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 17:24:17.085132 (+ 548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48000: BlockingRecv error: recv got EOF from 127.0.0.1:48000 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 17:24:17.205549 13516 catalog_manager.cc:1259] Loaded cluster ID: d4b08014c31b40488c8d9ad7156ff1f0
I20260811 17:24:17.205638 13516 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 656f43309a62496d9fcc5fb0a4cc419c: loading cluster ID for follower catalog manager: success
I20260811 17:24:17.207309 13516 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 656f43309a62496d9fcc5fb0a4cc419c: acquiring CA information for follower catalog manager: success
I20260811 17:24:17.208475 13516 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 656f43309a62496d9fcc5fb0a4cc419c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 17:24:17.234185 13544 catalog_manager.cc:1259] Loaded cluster ID: d4b08014c31b40488c8d9ad7156ff1f0
I20260811 17:24:17.234290 13544 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1135aa52eee74c679eaa179071c05a76: loading cluster ID for follower catalog manager: success
I20260811 17:24:17.235985 13544 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1135aa52eee74c679eaa179071c05a76: acquiring CA information for follower catalog manager: success
I20260811 17:24:17.237125 13544 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1135aa52eee74c679eaa179071c05a76: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 17:24:17.603721 13715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:17.599551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48012 (local address 127.9.157.189:44529)
0811 17:24:17.599846 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:17.599850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:17.599933 (+ 83us) server_negotiation.cc:407] Connection header received
0811 17:24:17.600058 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:17.600060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:17.600203 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:17.600309 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:24:17.600568 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:17.601093 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:17.601996 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:17.602256 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:17.602388 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:17.602713 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:24:17.603187 (+ 474us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:24:17.603302 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:17.603391 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 17:24:17.603449 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":107,"threads_started":1}
I20260811 17:24:17.611155 13726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:17.607153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42828 (local address 127.9.157.188:43545)
0811 17:24:17.607409 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:17.607413 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:17.607426 (+ 13us) server_negotiation.cc:407] Connection header received
0811 17:24:17.607511 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:17.607513 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:17.607654 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:17.607755 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:24:17.608006 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:17.608523 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:17.609450 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:17.609688 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:17.609789 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:17.610138 (+ 349us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:24:17.610584 (+ 446us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:24:17.610681 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:17.610884 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 17:24:17.610937 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":102,"threads_started":1}
I20260811 17:24:17.618451 13715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:17.614252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48018 (local address 127.9.157.189:44529)
0811 17:24:17.614651 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:17.614655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:17.615139 (+ 484us) server_negotiation.cc:407] Connection header received
0811 17:24:17.615236 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:17.615242 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:17.615313 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:17.615387 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:24:17.615826 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:17.616354 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:17.617355 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:17.617587 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:17.617701 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:17.617973 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:24:17.618048 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:24:17.618126 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:17.618230 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 17:24:17.618283 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308}
I20260811 17:24:17.619139 13726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:17.614496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42830 (local address 127.9.157.188:43545)
0811 17:24:17.614636 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:17.614640 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:17.614917 (+ 277us) server_negotiation.cc:407] Connection header received
0811 17:24:17.615026 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:17.615030 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:17.615123 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:17.615193 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:24:17.615528 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:17.616355 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:17.617739 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:17.618064 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:17.618138 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:17.618650 (+ 512us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:24:17.618726 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:24:17.618775 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:17.618944 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 17:24:17.618995 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 17:24:17.620098 13734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 17:24:17.614251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50872 (local address 127.9.157.190:42377)
0811 17:24:17.614535 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0811 17:24:17.614541 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 17:24:17.614792 (+ 251us) server_negotiation.cc:407] Connection header received
0811 17:24:17.614931 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 17:24:17.614935 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 17:24:17.615158 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 17:24:17.615282 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 17:24:17.615742 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:17.616522 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:17.617779 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 17:24:17.618120 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 17:24:17.618195 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 17:24:17.618993 (+ 798us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 17:24:17.619368 (+ 375us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 17:24:17.619464 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 17:24:17.619873 (+ 409us) server_negotiation.cc:299] Negotiation successful
0811 17:24:17.619942 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":82,"threads_started":1}
I20260811 17:24:17.626576 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 13128
I20260811 17:24:17.639564 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 13257
I20260811 17:24:17.650739 9846 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklEMhqg/build/debug/bin/kudu with pid 13377
2026-08-11T17:24:17Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3577 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (7124 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (313742 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST